Method and Apparatus for Decoding/Encoding a Video Signal

ABSTRACT

The present invention provides a method of decoding a video signal. The method includes the steps of obtaining view information of a picture from the video signal and generating information for reference picture management using the view information.

TECHNICAL FIELD

The present invention relates to a method for decoding/encoding a videosignal and apparatus thereof.

BACKGROUND ART

Compression encoding means a series of signal processing techniques fortransmitting digitalized information via communication circuit orstoring digitalized information in a form suitable for a storage medium.Objects for the compression encoding include audio, video, text, and thelike. In particular, a technique for performing compression encoding ona sequence is called video sequence compression. The video sequence isgenerally characterized in having spatial redundancy and temporalredundancy.

DISCLOSURE OF THE INVENTION Technical Object

The object of the present invention is to enhance coding efficiency of avideo signal.

Technical Solution

An object of the present invention is to code a video signal efficientlyby defining view information capable of identifying a view of picture.

Another object of the present invention is to code a video signalefficiently by providing a method of managing reference pictures usedfor inter-view prediction.

Another object of the present invention is to code a video signalefficiently by providing a method of constructing a reference picturelist for inter-view prediction.

Another object of the present invention is to code a video signalefficiently by providing a method of reordering a reference picture listfor inter-view prediction.

ADVANTAGEOUS EFFECTS

In coding a video signal, the present invention enables coding to bemore efficiently performed by providing a method of managing referencepictures used for inter-view prediction. And, the present inventionenables coding to be more efficiently performed by providing a method ofinitializing a reference picture list for inter-view prediction and amethod of reordering a reference picture list for inter-view prediction.In performing inter-view prediction using the present invention, it isable to improve a coding rate by reducing a load of DPB (decoded picturebuffer). And, the present invention enables more accurate prediction,thereby reducing the number of bits to be transmitted.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of an apparatus for decoding a videosignal according to the present invention.

FIG. 2 is a diagram of configuration information for a multi-view videoaddable to a multi-view video coded bit stream according to anembodiment of the present invention.

FIG. 3 is an internal block diagram of a reference picture listconstructing unit 620 according to an embodiment of the presentinvention.

FIG. 4 is a diagram of a hierarchical structure of level information forproviding view scalability of a video signal according to an embodimentof the present invention.

FIG. 5 is a diagram of a NAL-unit configuration including levelinformation within an extension area of a NAL header according to oneembodiment of the present invention.

FIG. 6 is a diagram of an overall predictive structure of a multi-viewvideo signal according to an embodiment of the present invention toexplain a concept of an inter-view picture group.

FIG. 7 is a diagram of a predictive structure according to an embodimentof the present invention to explain a concept of a newly definedinter-view picture group.

FIG. 8 is a schematic block diagram of an apparatus for decoding amulti-view video using inter-view picture group identificationinformation according to an embodiment of the present invention.

FIG. 9 is a flowchart of a process for constructing a reference picturelist according to an embodiment of the present invention.

FIG. 10 is a diagram to explain a method of initializing a referencepicture list when a current slice is a P-slice according to oneembodiment of the present invention.

FIG. 11 is a diagram to explain a method of initializing a referencepicture list when a current slice is a B-slice according to oneembodiment of the present invention.

FIG. 12 is an internal block diagram of a reference picture listreordering unit 630 according to an embodiment of the present invention.

FIG. 13 is an internal block diagram of a reference index assignmentchanging unit 643B or 645B according to one embodiment of the presentinvention.

FIG. 14 is a diagram to explain a process for reordering a referencepicture list using view information according to one embodiment of thepresent invention.

FIG. 15 is an internal block diagram of a reference picture listreordering unit 630 according to another embodiment of the presentinvention.

FIG. 16 is an internal block diagram of a reference picture listreordering unit 970 for inter-view prediction according to an embodimentof the present invention.

FIG. 17 and FIG. 18 are diagrams of syntax for reference picture listreordering according to one embodiment of the present invention.

FIG. 19 is a diagram of syntax for reference picture list reorderingaccording to another embodiment of the present invention.

FIG. 20 is a diagram for a process for obtaining an illuminationdifference value of a current block according to one embodiment of thepresent invention.

FIG. 21 is a flowchart of a process for performing illuminationcompensation of a current block according to an embodiment of thepresent invention.

FIG. 22 is a diagram of a process for obtaining an illuminationdifference prediction value of a current block using information for aneighboring block according to one embodiment of the present invention.

FIG. 23 is a flowchart of a process for performing illuminationcompensation using information for a neighboring block according to oneembodiment of the present invention.

FIG. 24 is a flowchart of a process for performing illuminationcompensation using information for a neighboring block according toanother embodiment of the present invention.

FIG. 25 is a diagram of a process for predicting a current picture usinga picture in a virtual view according to one embodiment of the presentinvention.

FIG. 26 is a flowchart of a process for synthesizing a picture in avirtual view in performing an inter-view prediction in MVC according toan embodiment of the present invention.

FIG. 27 is a flowchart of a method of executing a weighted predictionaccording to a slice type in video signal coding according to thepresent invention.

FIG. 28 is a diagram of macroblock types allowable in a slice type invideo signal coding according to the present invention.

FIG. 29 and FIG. 30 are diagrams of syntax for executing a weightedprediction according to a newly defined slice type according to oneembodiment of the present invention.

FIG. 31 is a flowchart of a method of executing a weighted predictionusing flag information indicating whether to execute inter-view weightedprediction in video signal coding according to the present invention.

FIG. 32 is a diagram to explain a weighted prediction method accordingto flag information indicating whether to execute a weighted predictionusing information for a picture in a view different from that of acurrent picture according to one embodiment of the present invention.

FIG. 33 is a diagram of syntax for executing a weighted predictionaccording to a newly defined flag information according to oneembodiment of the present invention.

FIG. 34 is a flowchart of a method of executing a weighted predictionaccording to a NAL (network abstraction layer) unit type according to anembodiment of the present invention.

FIG. 35 and FIG. 36 are diagrams of syntax for executing a weightedprediction in case that a NAL unit type is for multi-view video codingaccording to one embodiment of the present invention.

FIG. 37 is a partial block diagram of a video signal decoding apparatusaccording to a newly defined slice type according to an embodiment ofthe present invention.

FIG. 38 is a flowchart to explain a method of decoding a video signal inthe apparatus shown in FIG. 37 according to the present invention.

FIG. 39 is a diagram of a macroblock prediction mode according to oneembodiment of the present invention.

FIG. 40 and FIG. 41 are diagrams of syntax having slice type andmacroblock mode applied thereto according to the present invention.

FIG. 42 is a diagram of embodiments to which the slice types in FIG. 41are applied.

FIG. 43 is a diagram of various embodiments of the slice type includedin the slice types shown in FIG. 41.

FIG. 44 is a diagram of a macroblock allowable for a mixed slice type byprediction of two mixed predictions according to one embodiment of thepresent invention.

FIGS. 45 to 47 are diagrams of a macroblock type of a macroblockexisting in a mixed slice by prediction of two mixed predictionsaccording to one embodiment of the present invention.

FIG. 48 is a partial block diagram of a video signal encoding apparatusaccording to a newly defined slice type according to an embodiment ofthe present invention.

FIG. 49 is a flowchart of a method of encoding a video signal in theapparatus shown in FIG. 48 according to the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

To achieve these and other advantages and in accordance with the purposeof the present invention, as embodied and broadly described, a method ofdecoding a video signal includes the steps of obtaining view informationfor identifying a view of a picture from the video signal and generatinginformation for reference picture management using the view information.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, a method of decoding a video signalincludes the steps of obtaining view information for identifying a viewof a picture from the video signal and deriving a first variable forconstructing a reference picture list for inter-view prediction usingthe view information, wherein the reference picture list for theinter-view prediction is constructed using the first variable.

MODE FOR INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings.

The technique of compressing and encoding video signal data considersspatial redundancy, temporal redundancy, scalable redundancy, andinter-view redundancy. And, it is also able to perform a compressioncoding by considering a mutual redundancy between views in thecompression encoding process. The technique for the compression coding,which considers the inter-view redundancy, is just an embodiment of thepresent invention. And, the technical idea of the present invention isapplicable to the temporal redundancy, the scalable redundancy, etc.

Looking into a configuration of a bit stream in H.264/AVC, there existsa separate layer structure called a NAL (network abstraction layer)between a VCL (video coding layer) dealing with a moving pictureencoding process itself and a lower system that transports and storesencoded information. An output from an encoding process is VCL data andis mapped by NAL unit prior to transport or storage. Each NAL unitincludes compressed video data or RBSP (raw byte sequence payload:result data of moving picture compression) that is the datacorresponding to header information.

The NAL unit basically includes a NAL header and an RBSP. The NAL headerincludes flag information (nal_ref_idc) indicating whether a slice as areference picture of the NAL unit is included and an identifier(nal_unit_type) indicating a type of the NAL unit. Compressed originaldata is stored in the RBSP. And, RBSP trailing bit is added to a lastportion of the RBSP to represent a length of the RBSP as an 8-bitmultiplication. As the type of the NAL unit, there is IDR (instantaneousdecoding refresh) picture, SPS (sequence parameter set), PPS (pictureparameter set), SEI (supplemental enhancement information), or the like.

In the standardization, restrictions for various profiles and levels areset to enable implementation of a target product with an appropriatecost. In this case, a decoder should meet the restriction decidedaccording the corresponding profile and level. Thus, two concepts,‘profile’ and ‘level’ are defined to indicate a function or parameterfor representing how far the decoder can cope with a range of acompressed sequence. And, a profile indicator (profile_idc) can identifythat a bit stream is based on a prescribed profile. The profileindicator means a flag indicating a profile on which a bit stream isbased. For instance, in H.264/AVC, if a profile indicator is 66, itmeans that a bit stream is based on a baseline profile. If a profileindicator is 77, it means that a bit stream is based on a main profile.If a profile indicator is 88, it means that a bit stream is based on anextended profile. And, the profile identifier can be included in asequence parameter set.

So, in order to deal with a multi-view video, it needs to be identifiedwhether a profile of an inputted bit stream is a multi-view profile. Ifthe profile of the inputted bit stream is the multi-view profile, it isnecessary to add syntax to enable at least one additional informationfor multi-view to be transmitted. In this case, the multi-view profileindicates a profile mode handling multi-view video as an amendmenttechnique of H.264/AVC. In MVC, it may be more efficient to add syntaxas additional information for an MVC mode rather than unconditionalsyntax. For instance, when a profile indicator of AVC indicates amulti-view profile, if information for a multi-view video is added, itis able to enhance encoding efficiency.

A sequence parameter set indicates header information containinginformation crossing over coding of an overall sequence such as aprofile, a level, and the like. A whole compressed moving picture, i.e.,a sequence should begin at a sequence header. So, a sequence parameterset corresponding to header information should arrive at a decoderbefore data referring to the parameter set arrives. Namely, the sequenceparameter set RBSP plays a role as the header information for the resultdata of the moving picture compression. Once a bit stream is inputted, aprofile indicator preferentially identifies that the inputted bit streamis based on which one of a plurality of profiled. So, by adding a partfor deciding whether an inputted bit stream relates to a multi-viewprofile (e.g., ‘If (profile_idc==MULTI_VIEW_PROFILE)’) to syntax, it isdecided whether the inputted bit stream relates to the multi-viewprofile. Various kinds of configuration information can be added only ifthe inputted bit stream is approved as relating to the multi-viewprofile. For instance, it is able to add a number of total views, anumber of inter-view reference pictures (List0/1) in case of aninter-view picture group, a number of inter-view reference pictures(List0/1) in case of a non-inter-view picture group, and the like. And,various informations for view are usable for generation and managementof a reference picture list in a decoded picture buffer.

FIG. 1 is a schematic block diagram of an apparatus for decoding a videosignal according to the present invention.

Referring to FIG. 1, an apparatus for decoding a video signal accordingto the present invention includes a NAL parser 100, an entropy decodingunit 200, an inverse quantization/inverse transform unit 300, anintra-prediction unit 400, a deblocking filter unit 500, a decodedpicture buffer unit 600, an inter-prediction unit 700, and the like.

The decoded picture buffer unit 600 includes a reference picture storingunit 610, a reference picture list constructing unit 620, a referencepicture managing unit 650, and the like. And, the reference picture listconstructing unit 620 includes a variable deriving unit 625, a referencepicture list initializing unit 630, and a reference picture listreordering unit 640.

And, the inter-prediction unit 700 includes a motion compensation unit710, an illumination compensation unit 720, an illumination differenceprediction unit 730, a view synthesis prediction unit 740, and the like.

The NAL parser 100 carries out parsing by NAL unit to decode a receivedvideo sequence. In general, at least one sequence parameter set and atleast one picture parameter set are transferred to a decoder before aslice header and slice data are decoded. In this case, various kinds ofconfiguration informations can be included in a NAL header area or anextension area of a NAL header. Since MVC is an amendment technique fora conventional AVC technique, it may be more efficient to add theconfiguration informations in case of an MVC bit stream only rather thanunconditional addition. For instance, it is able to add flag informationfor identifying a presence or non-presence of an MVC bit stream in theNAL header area or the extension area of the NAL header. Only if aninputted bit stream is a multi-view video coded bit stream according tothe flag information, it is able to add configuration informations for amulti-view video. For instance, the configuration informations caninclude temporal level information, view level information, inter-viewpicture group identification information, view identificationinformation, and the like. This is explained in detail with reference toFIG. 2 as follows.

FIG. 2 is a diagram of configuration information for a multi-view videoaddable to a multi-view video coded bit stream according to oneembodiment of the present invention. Details of configurationinformation for a multi-view video are explained in the followingdescription.

First of all, temporal level information indicates information for ahierarchical structure to provide temporal scalability from a videosignal ({circle around (1)}). Through the temporal level information, itis able to provide a user with sequences on various time zones.

View level information indicates information for a hierarchicalstructure to provide view scalability from a video signal ({circlearound (2)}). In a multi-view video, it is necessary to define a levelfor a time and a level for a view to provide a user with varioustemporal and view sequences. In case of defining the above levelinformation, it is able to use temporal scalability and viewscalability. Hence, a user is able to select a sequence at a specifictime and view, or a selected sequence may be restricted by a condition.

The level informations can be set in various ways according to aspecific condition. For instance, the level information can be setdifferently according to camera location or camera alignment. And, thelevel information can be determined by considering view dependency. Forinstance, a level for a view having I-picture in an inter-view picturegroup is set to 0, a level for a view having P-picture in the inter-viewpicture group is set to 1, and a level for a view having B-picture inthe inter-view picture group is set to 2. Moreover, the levelinformations can be randomly set not based on a special condition. Theview level information will be explained in detail with reference toFIG. 4 and FIG. 5 later.

Inter-view picture group identification information indicatesinformation for identifying whether a coded picture of a current NALunit is an inter-view picture group ({circle around (3)}). In this case,the inter-view picture group means a coded picture in which all slicesreference only slices with the same picture order count. For instance,the inter-view picture group means an coded picture that refers toslices in a different view only without referring to slices in a currentview. In a decoding process of a multi-view video, an inter-view randomaccess may be needed. The inter-view picture group identificationinformation may be necessary to realize an efficient random access. And,inter-view reference information may be necessary for inter-viewprediction. So, inter-view picture group identification information canbe used to obtain the inter-view reference information. Moreover, theinter-view picture group identification information can be used to addreference pictures for inter-view prediction in constructing a referencepicture list. Besides, the inter-view picture group identificationinformation can be used to manage the added reference pictures for theinter-view prediction. For instance, the reference pictures may beclassified into inter-view picture groups and non-inter-view picturegroups and the classified reference pictures can be then marked that thereference pictures failing to be used for the inter-view predictionshall not be used. Meanwhile, the inter-view picture groupidentification information is applicable to a hypothetical referencedecoder. Details of the inter-view picture group identificationinformation will be explained with reference to FIG. 6 later.

The view identification information means information for discriminatinga picture in a current view from a picture in a different view ({circlearound (4)}). In coding a video signal, POC (picture order count) or‘frame_num’ may be used to identify each picture. In case of amulti-view video sequence, inter-view prediction can be executed. So,identification information to discriminate a picture in a current viewfrom a picture in another view is needed. So, it is necessary to defineview identification information for identifying a view of a picture. Theview identification information can be obtained from a header area of avideo signal. For instance, the header area can be a NAL header area, anextension area of a NAL header, or a slice header area. Information fora picture in a view different from that of a current picture is obtainedusing the view identification information and it is able to decode thevideo signal using the information of the picture in the different view.The view identification information is applicable to an overallencoding/decoding process of the video signal. And, the viewidentification information can be applied to multi-view video codingusing the ‘frame_num’ that considers a view instead of considering aspecific view identifier.

Meanwhile, the entropy decoding unit 200 carries out entropy decoding ona parsed bit stream, and a coefficient of each macroblock, a motionvector, and the like are then extracted. The inversequantization/inverse transform unit 300 obtains a transformedcoefficient value by multiplying a received quantized value by aconstant and then transforms the coefficient value inversely toreconstruct a pixel value. Using the reconstructed pixel value, theintra-prediction unit 400 performs an intra prediction from a decodedsample within a current picture. Meanwhile, the deblocking filter unit500 is applied to each coded macroblock to reduce block distortion. Afilter smoothens a block edge to enhance an image quality of a decodedframe. A selection of a filtering process depends on boundary strengthand gradient of an image sample around a boundary. Pictures throughfiltering are outputted or stored in the decoded picture buffer unit 600to be used as reference pictures.

The decoded picture buffer unit 600 plays a role in storing or openingthe previously coded pictures to perform an inter prediction. In thiscase, to store the pictures in the decoded picture buffer unit 600 or toopen the pictures, ‘frame_num’ and POC (picture order count) of eachpicture are used. So, since there exist pictures in a view differentfrom that of a current picture among the previously coded pictures, viewinformation for identifying a view of a picture may be usable togetherwith the ‘frame_num’ and the POC. The decoded picture buffer unit 600includes the reference picture storing unit 610, the reference picturelist constructing unit 620, and the reference picture managing unit 650.The reference picture storing unit 610 stores pictures that will bereferred to for the coding of the current picture. The reference picturelist constructing unit 620 constructs a list of reference pictures forthe inter-picture prediction. In multi-view video coding, inter-viewprediction may be needed. So, if a current picture refers to a picturein another view, it may be necessary to construct a reference picturelist for the inter-view prediction. In this case, the reference picturelist constructing unit 620 can use information for view in generatingthe reference picture list for the inter-view prediction. Details of thereference picture list constructing unit 620 will be explained withreference to FIG. 3 later.

FIG. 3 is an internal block diagram of a reference picture listconstructing unit 620 according to an embodiment of the presentinvention.

The reference picture list constructing unit 620 includes the variablederiving unit 625, the reference picture list initializing unit 630, andthe reference list reordering unit 640.

The variable deriving unit 625 derives variables used for referencepicture list initialization. For instance, the variable can be derivedusing ‘frame_num’ indicating a picture identification number. Inparticular, variables FrameNum and FrameNumWrap may be usable for eachshort-term reference picture. First of all, the variable FrameNum isequal to a value of a syntax element frame_num. The variableFrameNumWrap can be used for the decoded picture buffer unit 600 toassign a small number to each reference picture. And, the variableFrameNumWrap can be derived from the variable FrameNum. So, it is ableto derive a variable PicNum using the derived variable FrameNumWrap. Inthis case, the variable PicNum can mean an identification number of apicture used by the decoded picture buffer unit 600. In case ofindicating a long-term reference picture, a variable LongTermPicNum canbe usable.

In order to construct a reference picture list for inter-viewprediction, it is able to derive a first variable (e.g., ViewNum) toconstruct a reference picture list for inter-view prediction. Forinstance, it is able to derive a second variable (e.g., ViewId) using‘view_id’ for identifying a view of a picture. First of all, the secondvariable can be equal to a value of the syntax element ‘view_id’. And, athird variable (e.g., ViewIdWrap) can be used for the decoded picturebuffer unit 600 to assign a small view identification number to eachreference picture and can be derived from the second variable. In thiscase, the first variable ViewNum can mean a view identification numberof picture used by the decoded picture buffer unit 600. Yet, since anumber of reference pictures used for inter-view prediction inmulti-view video coding may be relatively smaller than that used fortemporal prediction, it may not define another variable to indicate aview identification number of a long-term reference picture.

The reference picture list initializing unit 630 initializes a referencepicture list using the above-mentioned variables. In this case, aninitialization process for the reference picture list may differaccording to a slice type. For instance, in case of decoding a P-slice,it is able to assign a reference index based on a decoding order. Incase of decoding a B-slice, it is able to assign a reference index basedon a picture output order. In case of initializing a reference picturelist for inter-view prediction, it is able to assign an index to areference picture based on the first variable, i.e., the variablederived from view information.

The reference picture list reordering unit 640 plays a role in enhancinga compression efficiency by assigning a smaller index to a picturefrequently referred to in the initialized reference picture list. Thisis because a small bit is assigned if a reference index for encodinggets smaller.

And, the reference picture list reordering unit 640 includes a slicetype checking unit 642, a reference picture list-0 reordering unit 643,and a reference picture list-1 reordering unit 645. If an initializedreference picture list is inputted, the slice type checking unit 642checks a type of a slice to be decoded and then decides whether toreorder a reference picture list-0 or a reference picture list-1. So,the reference picture list-0/1 reordering unit 643,645 performsreordering of the reference picture list-0 if the slice type is not anI-slice and also performs reordering of the reference picture list-1additionally if the slice type is a B-slice. Thus, after an end of thereordering process, a reference picture list is constructed.

The reference picture list0/1 reordering unit 643, 645 includes anidentification information obtaining unit 643A,645A and a referenceindex assignment changing unit 643B,645B respectively. Theidentification information obtaining unit 643A,645A receivedidentification information (reordering_of_pic_nums_idc) indicating anassigning method of a reference index if reordering of a referencepicture list is carried out according to flag information indicatingwhether to execute the reordering of the reference picture list. And,the reference index assignment changing unit 643B,645B reorders thereference picture list by changing an assignment of a reference indexaccording to the identification information.

And, the reference picture list reordering unit 640 is operable byanother method. For instance, reordering can be executed by checking aNAL unit type transferred prior to passing through the slice typechecking unit 642 and then classifying the NAL unit type into a case ofMVC NAL and a case of non-MVC NAL.

The reference picture managing unit 650 manages reference pictures toexecute inter prediction more flexibly. For instance, a memorymanagement control operation method and a sliding window method areusable. This is to manage a reference picture memory and a non-referencepicture memory by unifying the memories into one memory and realize anefficient memory management with a small memory. In multi-view videocoding, since pictures in a view direction have the same picture ordercount, information for identifying a view of each of the pictures isusable in marking the pictures in a view direction. And, referencepictures managed in the above manner can be used by the inter-predictionunit 700.

The inter-prediction unit 700 carries out inter prediction usingreference pictures stored in the decoded picture buffer unit 600. Aninter-coded macroblock can be divided into macroblock partitions. And,each of the macroblock partitions can be predicted from one or tworeference pictures. The inter-prediction unit 700 includes the motioncompensation unit 710, the illumination compensation unit 720, theillumination difference prediction unit 730, the view synthesisprediction unit 740, the weighted prediction unit 750, and the like.

The motion compensation unit 710 compensates for a motion of a currentblock using informations transferred from the entropy decoding unit 200.Motion vectors of neighboring blocks of the current block are extractedfrom a video signal, and then a motion vector predictor of the currentblock are derived from the motion vectors of the neighboring blocks.And, the motion of the current block is compensated using the derivedmotion vector predictor and a differential motion vector extracted fromthe video signal. And, it is able to perform the motion compensationusing one reference picture or a plurality of pictures. In multi-viewvideo coding, in case that a current picture refers to pictures indifferent views, it is able to perform motion compensation usingreference picture list information for the inter-view prediction storedin the decoded picture buffer unit 600. And, it is also able to performmotion compensation using vie information for identifying a view of thereference picture. A direct mode is an coding mode for predicting motioninformation of a current block from motion information for an encodedblock. Since this method is able to save a number of bits required forcoding the motion information, compression efficiency is enhanced. Forinstance, a temporal direction mode predicts motion information for acurrent block using a correlation of motion information in a temporaldirection. Using a method similar to this method, the present inventionis able to predict motion information for a current block using acorrelation of motion information in a view direction.

Meanwhile, in case that an inputted bit stream corresponds to amulti-view video, since the respective view sequences are obtained bydifferent camera, an illumination difference is generated by internaland external factors of the cameras. To prevent this, the illuminationcompensation unit 720 compensates the illumination difference. Inperforming the illumination compensation, it is able to use flaginformation indicating whether to perform illumination compensation on aspecific layer of a video signal. For instance, it is able to perform anillumination compensation using flag information indicating whether toperform the illumination compensation on a corresponding slice ormacroblock. In performing the illumination compensation using the flaginformation, the illumination compensation is applicable to variousmacroblock types (e.g., inter 16×16 mode, B-skip mode, direct mode,etc.).

In performing illumination compensation, it is able to use informationfor a neighboring block or information for a block in a view differentfrom that of a current block to reconstruct the current block. And, itis also able to use an illumination difference value of the currentblock. In this case, if the current block refers to blocks in adifferent view, it is able to perform illumination compensation usingthe reference picture list information for the inter-view predictionstored in the decoded picture buffer unit 600. In this case, theillumination difference value of the current block indicates adifference between an average pixel value of the current block and anaverage pixel value of a reference block corresponding to the currentblock. For example of using the illumination difference value, theillumination difference prediction value of the current block isobtained using neighboring blocks of the current block and a differencevalue (illumination difference residual) between the illuminationdifference value and the illumination difference prediction value isused. Hence, the decoding unit is able to reconstruct the illuminationdifference value of the current block using the illumination differenceresidual and the illumination difference prediction value. In obtainingan illumination difference prediction value of a current block, it isable to use information for a neighboring block. For instance, it isable to predict an illumination difference value of a current blockusing an illumination difference value of a neighbor block. Prior to theprediction, it is checked whether a reference index of the current blockis equal to that of the neighboring block. According to a result of thechecking, it is then decided what kind of a neighboring block or a valuewill be used.

The view synthesis prediction unit 740 is used to synthesize pictures ina virtual view using pictures in a view neighbor to a view of a currentpicture and predict the current picture using the synthesized picturesin the virtual view. The decoding unit is able to decide whether tosynthesize a picture in a virtual view according to an inter-viewsynthesis prediction identifier transferred from an encoding unit. Forinstance, if view synthesize_pred_flag=1 or view_syn_pred_flag=1, aslice or macroblock in a virtual view is synthesized. In this case, whenthe inter-view synthesis prediction identifier informs that a virtualview will be generated, it is able to generate a picture in the virtualview using view information for identifying a view of picture. And, inpredicting a current picture from the synthesized pictures in thevirtual view, it is able to use the view information to use the picturein the virtual view as a reference picture.

The weighted prediction unit 750 is used to compensate for a phenomenonthat an image quality of a sequence is considerably degraded in case ofencoding the sequence of which brightness temporarily varies. In MVC,weighted prediction can be performed to compensate for a brightnessdifference from a sequence in a different view as well as it isperformed for a sequence of which brightness temporarily varies. Forinstance, the weighted prediction method can be classified into explicitweighted prediction method and implicit weighted prediction method.

In particular, the explicit weighted prediction method can use onereference picture or two reference pictures. In case of using onereference picture, a prediction signal is generated from multiplying aprediction signal corresponding to motion compensation by a weightcoefficient. In case of using two reference pictures, a predictionsignal is generated from adding an offset value to a value resultingfrom multiplying a prediction signal corresponding to motioncompensation by a weight coefficient.

And, the implicit weighted prediction performs a weighted predictionusing a distance from a reference picture. As a method of obtaining thedistance from the reference picture, it is able to use POC (pictureorder count) indicating a picture output order for example. In thiscase, the POC may be obtained by considering identification of a view ofeach picture. In obtaining a weight coefficient for a picture in adifferent view, it is able to use view information for identifying aview of a picture to obtain a distance between views of the respectivepictures.

In video signal coding, depth information is usable for a specificapplication or another purpose. In this case, the depth information maymean information capable of indicating an inter-view disparitydifference. For instance, it is able to obtain a disparity vector byinter-view prediction. And, the obtained disparity vector should betransferred to a decoding apparatus for diparity compensation of acurrent block. Yet, if a depth map is obtained and then transferred tothe decoding apparatus, the disparity vector can be inferred from thedepth map (or disparity map) without transferring the disparity vectorto the decoding apparatus. In this case, it is advantageous in that thenumber of bits of depth information to be transferred to the decodingapparatus can be reduced. So, by deriving the disparity vector from thedepth map, it is able to provide a new disparity compensating method.Thus, in case of using a picture in a different view in the course ofderiving the disparity vector from the depth map, view information foridentifying a view of picture can be used.

The inter-predicted or intra-predicted pictures through theabove-explained process are selected according to a prediction mode toreconstruct a current picture. In the following description, variousembodiments providing an efficient decoding method of a video signal areexplained.

FIG. 4 is a diagram of a hierarchical structure of level information forproviding view scalability of a video signal according to one embodimentof the present invention.

Referring to FIG. 4, level information for each view can be decided byconsidering inter-view reference information. For instance, since it isimpossible to decode a P-picture and a B-picture without an I-picture,it is able to assign ‘level=0’ to a base view of which inter-viewpicture group is the I-picture, ‘level=1’ to a base view of whichinter-view picture group is the P-picture, and ‘level=2’ to a base viewof which inter-view picture group is the B-picture. Yet, it is also ableto decide level information randomly according to a specific standard.

Level information can be randomly decided according to a specificstandard or without a standard. For instance, in case that levelinformation is decided based on a view, it is able to set a view V0 as abase view to view level 0, a view of pictures predicted using picturesin one view to view level 1, and a view of pictures predicted usingpictures in a plurality of views to view level 2. In this case, at leastone view sequence to have compatibility with a conventional decoder(e.g., H.264/AVC, MPEG-2, MPEG-4, etc.) may be needed. This base viewbecomes a base of multi-view coding, which may correspond to a referenceview for prediction of another view. A sequence corresponding to a baseview in MVC (multi-view video coding) can be configured into anindependent bit stream by being encoded by a conventional sequenceencoding scheme (MPEG-2, MPEG-4, H.263, H.264, etc.). A sequencecorresponding to a base view is compatible with H.264/AVC or may not.Yet, a sequence in a view compatible with H.264/AVC corresponds to abase view.

As can be seen in FIG. 4, it is able to set a view V2 of picturespredicted using pictures in the view V0, a view V4 of pictures predictedusing pictures in the view V2, a view V6 of pictures predicted usingpictures in the view V4, and a view V7 of pictures predicted usingpictures in the view V6 to view level 1. And, it is able to set a viewV1 of pictures predicted using pictures in the views V0 and V2 and aview V3 predicted in the same manner, and a view V5 predicted in thesame manner to view level 2. So, in case that a user's decoder is unableto view a multi-view video sequence, it decodes sequences in the viewcorresponding to the view level 0 only. In case that the user's decoderis restricted by profile information, it is able to decode theinformation of a restricted view level only. In this case, a profilemeans that technical elements for algorithm in a video encoding/decodingprocess are standardized. In particular, the profile is a set oftechnical elements required for decoding a bit sequence of a compressedsequence and can be a sort of a sub-standardization.

According to another embodiment of the present invention, levelinformation may vary according to a location of a camera. For instance,assuming that views V0 and V1 are sequences obtained by a camera locatedin front, that views V2 and V3 are sequences located in rear, that viewsV4 and V5 are sequences located in left, and that views V6 and V7 aresequences located in right, it is able to set the views V0 and V1 toview level 0, the views V2 and V3 to view level 1, the views V4 and V5to view level 2, and the views V6 and V7 to view level 3. Alternatively,level information may vary according to camera alignment. Alternatively,level information can be randomly decided not based on a specificstandard.

FIG. 5 is a diagram of a NAL-unit configuration including levelinformation within an extension area of a NAL header according to oneembodiment of the present invention.

Referring to FIG. 5, a NAL unit basically includes a NAL header and anRBSP. The NAL header includes flag information (nal_ref_idc) indicatingwhether a slice becoming a reference picture of the NAL unit is includedand an identifier (nal_unit_type) indicating a type of the NAL unit.And, the NAL header may further include level information (view_level)indicating information for a hierarchical structure to provide viewscalability.

Compressed original data is stored in the RBSP, and RBSP trailing bit isadded to a last portion of the RBSP to represent a length of the RBSP asan 8-bit multiplication number. As the types of the NAL unit, there areIDR (instantaneous decoding refresh), SPS (sequence parameter set), PPS(picture parameter set), SEI (supplemental enhancement information),etc.

The NAL header includes information for a view identifier. And, a videosequence of a corresponding view level is decoded with reference to theview identifier in the course of performing decoding according to a viewlevel.

The NAL unit includes a NAL header 51 and a slice layer 53. The NALheader 51 includes a NAL header extension 52. And, the slice layer 53includes a slice header 54 and a slice data 55.

The NAL header 51 includes an identifier (nal_unit_type) indicating atype of the NAL unit. For instance, the identifier indicating the NALunit type may be an identifier for both scalable coding and multi-viewvideo coding. In this case, the NAL header extension 52 can include flaginformation discriminating whether a current NAL is the NAL for thescalable video coding or the NAL for the multi-view video coding. And,the NAL header extension 52 can include extension information for thecurrent NAL according to the flag information. For instance, in casethat the current NAL is the NAL for the multi-view video codingaccording to the flag information, the NAL header extension 52 caninclude level information (view_level) indicating information for ahierarchical structure to provide view scalability.

FIG. 6 is a diagram of an overall predictive structure of a multi-viewvideo signal according to one embodiment of the present invention toexplain a concept of an inter-view picture group.

Referring to FIG. 6, T0 to T100 on a horizontal axis indicate framesaccording to time and S0 to S7 on a vertical axis indicate framesaccording to view. For instance, pictures at T0 mean frames captured bydifferent cameras on the same time zone T0, while pictures at S0 meansequences captured by a single camera on different time zones. And,arrows in the drawing indicate predictive directions and predictiveorders of the respective pictures. For instance, a picture P0 in a viewS2 on a time zone T0 is a picture predicted from I0, which becomes areference picture of a picture P0 in a view S4 on the time zone T0. And,it becomes a reference picture of pictures B1 and B2 on time zones T4and T2 in the view S2, respectively.

In a multi-view video decoding process, an inter-view random access maybe needed. So, an access to a random view should be possible byminimizing the decoding effort. In this case, a concept of an inter-viewpicture group may be needed to realize an efficient access. Theinter-view picture group means a coded picture in which all slicesreference only slices with the same picture order count. For instance,the inter-view picture group means an coded picture that refers toslices in a different view only without referring to slices in a currentview. In FIG. 6, if a picture I0 in a view S0 on a time zone T0 is aninter-view picture group, all pictures in different views on the sametime zone, i.e., the time zone T0, become inter-view picture groups. Foranother instance, if a picture 10 in a view S0 on a time zone T8 is aninter-view picture group, all pictures in different views on the sametime zone, i.e., the time zone T8, are inter-view picture groups.Likewise, all pictures in T16, . . . , T96, and T100 become inter-viewpicture groups as well.

FIG. 7 is a diagram of a predictive structure according to an embodimentof the present invention to explain a concept of a newly definedinter-view picture group.

In an overall predictive structure of MVC, GOP can begin with anI-picture. And, the I-picture is compatible with H.264/AVC. So, allinter-view picture groups compatible with H.264/AVC can always becomethe I-picture. Yet, in case that the I-pictures are replaced by aP-picture, more efficient coding is enabled. In particular, moreefficient coding is enabled using the predictive structure enabling GOPto begin with the P-picture compatible with H.264/AVC.

In this case, if the inter-view picture group is re-defined, all slicesbecome encoded picture capable of referring to not only a slice in aframe on a same time zone but also a slice in the same view on adifferent time zone. Yet, in case of referring to a slice on a differenttime zone in a same view, it can be restricted to the inter-view picturegroup compatible with H.264/AVC only. For instance, a P-picture on atiming point T8 in a view S0 in FIG. 6 can become a newly definedinter-view picture group. Likewise, a P-picture on a timing point T96 ina view S0 or a P-picture on a timing point T100 in a view S0 can becomea newly defined inter-view picture group. And, the inter-view picturegroup can be defined only if it is a base view.

After the inter-view picture group has been decoded, all of thesequentially coded pictures are decoded from pictures decoded ahead ofthe inter-view picture group in an output order withoutinter-prediction.

Considering the overall coding structure of the multi-view video shownin FIG. 6 and FIG. 7, since inter-view reference information of aninter-view picture group differs from that of a non-inter-view picturegroup, it is necessary to discriminate the inter-view picture group andthe non-inter-view picture group from each other according to theinter-view picture group identification information.

The inter-view reference information means the information capable ofrecognizing a predictive structure between inter-view pictures. This canbe obtained from a data area of a video signal. For instance, it can beobtained from a sequence parameter set area. And, the inter-viewreference information can be recognized using the number of referencepictures and view information for the reference pictures. For instance,the number of total views is obtained and the view information foridentifying each view can be then obtained based on the number of thetotal views. And, it is able to obtain the number of the referencepictures for a reference direction for each view. According to thenumber of the reference pictures, it is able to obtain the viewinformation for each of the reference pictures. In this manner, theinter-view reference information can be obtained. And, the inter-viewreference information can be recognized by discriminating an inter-viewpicture group and a non-inter-view picture group. This can be recognizedusing inter-view picture group identification information indicatingwhether a coded slice in a current NAL is an inter-view picture group.Details of the inter-view picture group identification information areexplained with reference to FIG. 8 as follows.

FIG. 8 is a schematic block diagram of an apparatus for decoding amulti-view video using inter-view picture group identifying informationaccording to one embodiment of the present invention.

Referring to FIG. 8, a decoding apparatus according to one embodiment ofthe present invention includes a bit stream deciding unit 81, aninter-view picture group identification information obtaining unit 82,and a multi-view video decoding unit 83.

If a bit stream is inputted, the bit stream deciding unit 81 decideswhether the inputted bit stream is a coded bit stream for a scalablevideo coding or a coded bit stream for multi-view video coding. This canbe decided by flag information included in the bit stream.

The inter-view picture group identification information obtaining unit82 is able to obtain inter-view picture group identification informationif the inputted bit stream is the bit stream for a multi-view videocoding as a result of the decision. If the obtained inter-view picturegroup identification information is ‘true’, it means that a coded sliceof a current NAL is an inter-view picture group. If the obtainedinter-view picture group identification information is ‘false’, it meansthat a coded slice of a current NAL is a non-inter-view picture group.The inter-view picture group identification information can be obtainedfrom an extension area of a NAL header or a slice layer area.

The multi-view video decoding unit 83 decodes a multi-view videoaccording to the inter-view picture group identification information.According to an overall coding structure of a multi-view video sequence,inter-view reference information of an inter-view picture group differsfrom that of a non-inter-view picture group. So, it is able to use theinter-view picture group identification information in adding referencepictures for inter-view prediction to generate a reference picture listfor example. And, it is also able to use the inter-view picture groupidentification information to manage the reference pictures for theinter-view prediction. Moreover, the inter-view picture groupidentification information is applicable to a hypothetical referencedecoder.

As another example of using the inter-view picture group identificationinformation, in case of using information in a different view for eachdecoding process, inter-view reference information included in asequence parameter set is usable. In this case, information fordiscriminating whether a current picture is an inter-view picture groupor a non-inter-view picture group, i.e., inter-view picture groupidentification information may be required. So, it is able to usedifferent inter-view reference information for each decoding process.

FIG. 9 is a flowchart of a process for generating a reference picturelist according to an embodiment of the present invention.

Referring to FIG. 9, the decoded picture buffer unit 600 plays role instoring or opening previously coded pictures to perform inter-pictureprediction.

First of all, pictures coded prior to a current picture are stored inthe reference picture storing unit 610 to be used as reference pictures(S91).

In multi-view video coding, since some of the previously coded picturesare in a view different from that of the current picture, viewinformation for identifying a view of a picture can be used to utilizethese pictures as reference pictures. So, the decoder should obtain viewinformation for identifying a view of a picture (S92). For instance, theview information can include ‘view id’ for identifying a view of apicture.

The decoded picture buffer unit 600 needs to derive a variable usedtherein to generate a reference picture list. Since inter-viewprediction may be required for multi-view video coding, if a currentpicture refers to a picture in a different view, it may be necessary togenerate a reference picture list for inter-view prediction. In thiscase, the decoded picture buffer unit 600 needs to derive a variableused to generate the reference picture list for the inter-viewprediction using the obtained view information (S93).

A reference picture list for temporal prediction or a reference picturelist for inter-view prediction can be generated by a different methodaccording to a slice type of a current slice (S94). For instance, if aslice type is a P/SP slice, a reference picture list 0 is generated(S95). In case that a slice type is a B-slice, a reference picture list0 and a reference picture list 1 are generated (S96). In this case, thereference picture list 0 or 1 can include the reference picture list forthe temporal prediction only or both of the reference picture list forthe temporal prediction and the reference picture list for theinter-view prediction. This will be explained in detail with referenceto FIG. 8 and FIG. 9 later.

The initialized reference picture list undergoes a process for assigninga smaller number to a frequently referred picture to further enhance acompression rate (S97). And, this can be called a reordering process fora reference picture list, which will be explained in detail withreference to FIGS. 12 to 19 later. The current picture is decoded usingthe reordered reference picture list and the decoded picture buffer unit600 needs to manage the decoded reference pictures to operate a buffermore efficiently (S98). The reference pictures managed by the aboveprocess are read by the inter-prediction unit 700 to be used forinter-prediction. In multi-view video coding, the inter-prediction caninclude inter-view prediction. In this case, the reference picture listfor the inter-view prediction is usable.

Detailed examples for a method of generating a reference picture listaccording to a slice type are explained with reference to FIG. 10 andFIG. 11 as follows.

FIG. 10 is a diagram to explain a method of initializing a referencepicture list when a current slice is a P-slice according to oneembodiment of the present invention.

Referring to FIG. 10, a time is indicated by T0, T1, . . . , TN, while aview is indicated by V0, V1, . . . , V4. For instance, a current pictureindicates a picture at a time T3 in a view V4. And, a slice type of thecurrent picture is a P-slice. ‘PN’ is an abbreviation of a variablePicNum, ‘LPN’ is an abbreviation of a variable LongTermPicNum, and ‘VN’is an abbreviation of a variable ViewNum. A numeral attached to an endportion of each of the variables indicates an index indicating a time ofeach picture (for PN or LPN) or a view of each picture (for VN). This isapplicable to FIG. 11 in the same manner.

A reference picture list for temporal prediction or a reference picturelist for inter-view prediction can be generated in a different wayaccording to a slice type of a current slice. For instance, a slice typein FIG. 12 is a P/SP slice. In this case, a reference picture list 0 isgenerated. In particular, the reference picture list 0 can include areference picture list for temporal prediction and/or a referencepicture list for inter-view prediction. In the present embodiment, it isassumed that a reference picture list includes both a reference picturelist for temporal prediction and a reference picture list for inter-viewprediction.

There are various methods for ordering reference pictures. For instance,reference pictures can be aligned according to in order of decoding orpicture output. Alternatively, reference pictures can be aligned basedon a variable derived using view information. Alternatively, referencepictures can be aligned according to inter-view reference informationindicating an inter-view prediction structure.

In case of a reference picture list for temporal prediction, short-termreference pictures and long-term reference pictures can be aligned basedon a decoding order. For instance, they can be aligned according to avalue of a variable PicNum or LongTermPicNum derived from a valueindicating a picture identification number (e.g., frame_num orLongtermframeidx). First of all, short-term reference pictures can beinitialized prior to long-term reference pictures. An order of aligningthe short-term reference pictures can be set from a reference picturehaving a highest value of variable PicNum to a reference picture havinga lowest variable value. For instance, the short-term reference picturescan be aligned in order of PN1 having a highest variable, PN2 having anintermediate variable, and PN0 having a lowest variable among PN0 toPN2. An order of aligning the long-term reference pictures can be setfrom a reference picture having a lowest value of variableLongTermPicNum to a reference picture having a highest variable value.For instance, the long-term reference pictures can be aligned in orderof LPN0 having a highest variable and LPN1 having a lowest variable.

In case of a reference picture list for inter-view prediction, referencepictures can be aligned based on a first variable ViewNum derived usingview information. In particular, reference pictures can be aligned inorder of a reference picture having a highest first variable (ViewNum)value to a reference picture having a lowest first variable (ViewNum)value. For instance, reference pictures can be aligned in order of VN3having a highest variable, VN2, VN1, and VN0 having a lowest variableamong VN0, VN1, VN2, and VN3.

Thus, both of the reference picture list for the temporal prediction andthe reference picture list for the inter-view prediction can be managedas one reference picture list. Alternatively, both of the referencepicture list for the temporal prediction and the reference picture listfor the inter-view prediction can be managed as separate referencepicture lists, respectively. In case of managing both of the referencepicture list for the temporal prediction and the reference picture listfor the inter-view prediction as one reference picture list, they can beinitialized according to an order or simultaneously. For instance, incase of initializing both of the reference picture list for the temporalprediction and the reference picture list for the inter-view predictionaccording to an order, the reference picture list for the temporalprediction is preferentially initialized and the reference picture listfor the inter-view prediction is then initialized in addition. Thisconcept is applicable to FIG. 11 as well.

A case that a slice type of a current picture is a B-slice is explainedwith reference to FIG. 11 as follows.

FIG. 11 is a diagram to explain a method o initializing a referencepicture list when a current slice is a B-slice according to oneembodiment of the present invention.

Referring to FIG. 9, in case that a slice type is a B-slice, a referencepicture list 0 and a reference picture list 1 are generated. In thiscase, the reference picture list 0 or the reference picture list 1 caninclude a reference picture list for temporal prediction only or both areference picture list for temporal prediction and a reference picturelist for inter-view prediction.

In case of the reference picture list for the temporal prediction, ashort-term reference picture aligning method may differ from a long-termreference picture aligning method. For instance, in case of short-termreference pictures, reference pictures can be aligned according to apicture order count (hereinafter abbreviated POC). In case of long-termreference pictures, reference pictures can be aligned according to avariable (LongtermPicNum) value. And, the short-term reference picturescan be initialized prior to the long-term reference pictures.

In order of aligning short-term reference pictures of the referencepicture list 0, reference pictures are preferentially aligned from areference picture having a highest POC value to a reference picturehaving a lowest POC value among reference pictures having POC valuessmaller than that of a current picture, and then aligned from areference picture having a lowest POC value to a reference picturehaving a highest POC value among reference pictures having POC valuesgreater than that of the current picture. For instance, referencepictures can be preferentially aligned from PN1 having a highest POCvalue in reference pictures PN0 and PN1 having POC values smaller thanthat of a current picture to PN0, and then aligned from PN3 having alowest POC value in reference pictures PN3 and PN4 having a POC valuesmaller than that of a current picture to PN4.

In order of aligning long-term reference pictures of the referencepicture list 0, reference pictures are aligned from a reference picturehaving a lowest variable LongtermPicNum to a reference picture having ahighest variable. For instance, reference pictures are aligned from LPN0having a lowest value in LPN0 and LPN1 to LPN1 having a second lowestvariable.

In case of the reference picture list for the inter-view prediction,reference pictures can be aligned based on a first variable ViewNumderived using view information. For instance, in case of the referencepicture list 0 for the inter-view prediction, reference pictures can bealigned from a reference picture having a highest first variable valueamong reference pictures having first variable values lower than that ofa current picture to a reference picture having a lowest first variablevalue. The reference pictures are then aligned from a reference picturehaving a lowest first variable value among reference pictures havingfirst variable values greater than that of the current picture to areference picture having a highest first variable value. For instance,reference pictures are preferentially aligned from VN1 having a highestfirst variable value in VN0 and VN1 having first variable values smallerthan that of a current picture to VN0 having a lowest first variablevalue and then aligned from VN3 having a lowest first variable value inVN3 and VN4 having first variable values greater than that of thecurrent picture to VN4 having a highest first variable value.

In case of the reference picture list 1, the above-explained aligningmethod of the reference list 0 is similarly applicable.

First of all, in case of the reference picture list for the temporalprediction, in order of aligning short-term reference pictures of thereference picture list 1, reference pictures are preferentially alignedfrom a reference picture having a lowest POC value to a referencepicture having a highest POC value among reference pictures having POCvalues greater than that of a current picture and then aligned from areference picture having a highest POC value to a reference picturehaving a lowest POC value among reference pictures having POC valuessmaller than that of the current picture. For instance, referencepictures can be preferentially aligned from PN3 having a lowest POCvalue in reference pictures PN3 and PN4 having POC values greater thanthat of a current picture to PN4 and then aligned from PN1 having ahighest POC value in reference pictures PN0 and PN1 having POC valuesgreater than that of the current picture to PN0.

In order of aligning long-term reference pictures of the referencepicture list 1, reference pictures are aligned from a reference picturehaving a lowest variable LongtermPicNum to a reference picture having ahighest variable. For instance, reference pictures are aligned from LPN0having a lowest value in LPN0 and LPN1 to LPN1 having a lowest variable.

In case of the reference picture list for the inter-view prediction,reference pictures can be aligned based on a first variable ViewNumderived using view information. For instance, in case of the referencepicture list 1 for the inter-view prediction, reference pictures can bealigned from a reference picture having a lowest first variable valueamong reference pictures having first variable values greater than thatof a current picture to a reference picture having a highest firstvariable value.

The reference pictures are then aligned from a reference picture havinga highest first variable value among reference pictures having firstvariable values smaller than that of the current picture to a referencepicture having a lowest first variable value. For instance, referencepictures are preferentially aligned from VN3 having a lowest firstvariable value in VN3 and VN4 having first variable values greater thanthat of a current picture to VN4 having a highest first variable valueand then aligned from VN1 having a highest first variable value in VN0and VN1 having first variable values smaller than that of the currentpicture to VN0 having a lowest first variable value.

The reference picture list initialized by the above process istransferred to the reference picture list reordering unit 640. Theinitialized reference picture list is then reordered for more efficientcoding. The reordering process is to reduce a bit rate by assigning asmall number to a reference picture having highest probability in beingselected as a reference picture by operating a decoded picture buffer.Various methods of reordering a reference picture list are explainedwith reference to FIGS. 12 to 19 as follows.

FIG. 12 is an internal block diagram of the reference picture listreordering unit 640 according to one embodiment of the presentinvention.

Referring to FIG. 12, the reference picture list reordering unit 640basically includes a slice type checking unit 642, a reference picturelist 0 reordering unit 643, and a reference picture list 1 reorderingunit 645.

In particular, the reference picture list 0 reordering unit 643 includesa first identification information obtaining unit 643A, and a firstreference index assignment changing unit 643B. And, the referencepicture list 1 reordering unit 645 includes a second identificationobtaining unit 645A and a second reference index assignment changingunit 645B.

The slice type checking unit 642 checks a slice type of a current slice.It is then decided whether to reorder a reference picture list 0 and/ora reference picture list 1 according to the slice type. For instance, ifa slice type of a current slice is an I-slice, both of the referencepicture list 0 and the reference picture list 1 are not reordered. If aslice type of a current slice is a P-slice, the reference picture list 0is reordered only. If a slice type of a current slice is a B-slice, bothof the reference picture list 0 and the reference picture list 1 arereordered.

The reference picture list 0 reordering unit 643 is activated if flaginformation for executing reordering of the reference picture list 0 is‘true’ and if the slice type of the current slice is not the I-slice.The first identification information obtaining unit 643A obtainsidentification information indicating a reference index assigningmethod. The first reference index assignment changing unit 643B changesa reference index assigned to each reference picture of the referencepicture list 0 according to the identification information.

Likewise, the reference picture list 1 reordering unit 645 is activatedif flag information for executing reordering of the reference picturelist 1 is ‘true’ and if the slice type of the current slice is theB-slice. The second identification information obtaining unit 645Aobtains identification information indicating a reference indexassigning method. The second reference index assignment changing unit645B changes a reference index assigned to each reference picture of thereference picture list 1 according to the identification information.

So, reference picture list information used for actual inter-predictionis generated through the reference picture list 0 reordering unit 643and the reference picture list 1 reordering unit 645.

A method of changing a reference index assigned to each referencepicture by the first or second reference index assignment changing unit643B or 645B is explained with reference to FIG. 13 as follows.

FIG. 13 is an internal block diagram of a reference index assignmentchanging unit 643B or 645B according to one embodiment of the presentinvention. In the following description, the reference picture list 0reordering unit 643 and the reference picture list 1 reordering unit 645shown in FIG. 12 are explained together.

Referring to FIG. 13, each of the first and second reference indexassignment changing units 643B and 645B includes a reference indexassignment changing unit for temporal prediction 644A, a reference indexassignment changing unit for long-term reference picture 644B, areference index assignment changing unit for inter-view prediction 644C,and a reference index assignment change terminating unit 644D. Accordingto identification informations obtained by the first and secondidentification information obtaining units 643A and 645A, parts withinthe first and second reference index assignment changing units 643B and645B are activated, respectively. And, the reordering process keepsbeing executed until identification information for terminating thereference index assignment change is inputted.

For instance, if identification information for changing assignment of areference index for temporal prediction is received from the first orsecond identification information obtaining unit 643A or 645A, thereference index assignment changing unit for temporal prediction 644A isactivated. The reference index assignment changing unit for temporalprediction 644A obtains a picture number difference according to thereceived identification information. In this case, the picture numberdifference means a difference between a picture number of a currentpicture and a predicted picture number. And, the predicted picturenumber may indicate a number of a reference picture assigned rightbefore. So, it is able to change the assignment of the reference indexusing the obtained picture number difference. In this case, the picturenumber difference can be added/subtracted to/from the predicted picturenumber according to the identification information.

For another instance, if identification information for changingassignment of a reference index to a designated long-term referencepicture is received, the reference index assignment changing unit for along-term reference picture 644B is activated. The reference indexassignment changing unit for a long-term reference picture 644B obtainsa long-term reference picture number of a designated picture accordingto the identification number.

For another instance, if identification information for changingassignment of a reference index for inter-view prediction is received,the reference index assignment changing unit for inter-view prediction644C is activated. The reference index assignment changing unit forinter-view prediction 644C obtains view information difference accordingto the identification information. In this case, the view informationdifference means a difference between a view number of a current pictureand a predicted view number. And, the predicted view number may indicatea view number of a reference picture assigned right before. So, it isable to change assignment of a reference index using the obtained viewinformation difference. In this case, the view information differencecan be added/subtracted to/from the predicted view number according tothe identification information.

For another instance, if identification information for terminating areference index assignment change is received, the reference indexassignment change terminating unit 644D is activated. The referenceindex assignment change terminating unit 644D terminates an assignmentchange of a reference index according to the received identificationinformation. So, the reference picture list reordering unit 640generates reference picture list information.

Thus, reference pictures used for inter-view prediction can be managedtogether with reference pictures used for temporal prediction.Alternatively, reference pictures used for inter-view prediction can bemanaged separate from reference pictures used for temporal prediction.For this, new informations for managing the reference pictures used forthe inter-view prediction may be required. This will be explained withreference to FIGS. 15 to 19 later.

Details of the reference index assignment changing unit for inter-viewprediction 644C are explained with reference to FIG. 14 as follows.

FIG. 14 is a diagram to explain a process for reordering a referencepicture list using view information according to one embodiment of thepresent invention.

Referring to FIG. 14, if a view number VN of a current picture is 3, ifa size of a decoded picture buffer DPBsize is 4, and if a slice type ofa current slice is a P-slice, a reordering process for a referencepicture list 0 is explained as follows.

First of all, an initially predicted view number is ‘3’ that is the viewnumber of the current picture. And, an initial alignment of thereference picture list 0 for inter-view prediction is ‘4, 5, 6, 2’({circle around (1)}). In this case, if identification information forchanging assignment of a reference index for inter-view prediction bysubtracting a view information difference is received, ‘1’ is obtainedas the view information difference according to the receivedidentification information. A newly predicted view number (=2) iscalculated by subtracting the view information difference (=1) from thepredicted view number (=3). In particular, a first index of thereference picture list 0 for the inter-view prediction is assigned to areference picture having the view number 2. And, a picture previouslyassigned to the first index can be moved to a most rear part of thereference picture list 0. So, the reordered reference picture list 0 is‘2, 5, 6, 4’ ({circle around (2)}). Subsequently, if identificationinformation for changing assignment of a reference index for inter-viewprediction by subtracting the view information difference is received,‘−2’ is obtained as the view information difference according to theidentification information. A newly predicted view number (=4) is thencalculated by subtracting the view information difference (=−2) from thepredicted view number (=2). In particular, a second index of thereference picture list 0 for the inter-view prediction is assigned to areference picture having a view number 4. Hence, the reordered referencepicture list 0 is ‘2, 4, 6, 5’ ({circle around (3)}). Subsequently, ifidentification information for terminating the reference indexassignment change is received, the reference picture list 0 having thereordered reference picture list 0 as an end is generated according tothe received identification information ({circle around (4)}). Hence,the order of the finally generated reference picture list 0 for theinter-view prediction is ‘2, 4, 6, 5’.

For another instance of reordering the rest of the pictures after thefirst index of the reference picture list 0 for the inter-viewprediction has been assigned, a picture assigned to each index can bemoved to a position right behind that of the corresponding picture. Inparticular, a second index is assigned to a picture having a view number4, a third index is assigned to a picture (view number 5) to which thesecond index was assigned, and a fourth index is assigned to a picture(view number 6) to which the third index was assigned. Hence, thereordered reference picture list 0 becomes ‘2, 4, 5, 6’. And, asubsequent reordering process can be executed in the same manner.

The reference picture list generated by the above-explained process isused for inter-prediction. Both of the reference picture list for theinter-view prediction and the reference picture list for the temporalprediction can be managed as one reference picture list. Alternatively,each of the reference picture list for the inter-view prediction and thereference picture list for the temporal prediction can be managed as aseparate reference picture list. This is explained with reference toFIGS. 15 to 19 as follows.

FIG. 15 is an internal block diagram of a reference picture listreordering unit 640 according to another embodiment of the presentinvention.

Referring to FIG. 15, in order to manage a reference picture list forinter-view prediction as a separate reference picture list, newinformations may be needed. For instance, a reference picture list fortemporal prediction is reordered, and a reference picture list forinter-view prediction is then reordered in some cases.

The reference picture list reordering unit 640 basically includes areference picture list reordering unit for temporal prediction 910, aNAL type checking unit 960, and a reference picture list reordering unitfor inter-view prediction 970.

The reference picture list reordering unit for temporal prediction 910includes a slice type checking unit 642, a third identificationinformation obtaining unit 920, a third reference index assignmentchanging unit 930, a fourth identification information obtaining unit940, and a fourth reference index assignment changing unit 950. Thethird reference index assignment changing unit 930 includes a referenceindex assignment changing unit for temporal prediction 930A, a referenceindex assignment changing unit for a long-term reference picture 930B,and a reference index assignment change terminating unit 930C. Likewise,the fourth reference index assignment changing unit 950 includes areference index assignment changing unit for temporal prediction 950A, areference index assignment changing unit for long-term reference picture950B, and a reference index assignment change terminating unit 950C.

The reference picture list reordering unit for temporal prediction 910reorders reference pictures used for temporal prediction. Operations ofthe reference picture list reordering unit for temporal prediction 910are identical to those of the aforesaid reference picture listreordering unit 640 shown in FIG. 10 except informations for thereference pictures for the inter-view prediction. So, details of thereference picture list reordering unit for temporal prediction 910 areomitted in the following description.

The NAL type checking unit 960 checks a NAL type of a received bitstream. If the NAL type is a NAL for multi-view video coding, referencepictures used for the inter-view prediction are reordered by thereference picture list reordering unit for temporal prediction 970. Thegenerated reference picture list for the inter-view prediction are usedfor inter-prediction together with the reference picture list generatedby the reference picture list reordering unit for temporal prediction910. Yet, if the NAL type is not the NAL for the multi-view videocoding, the reference picture list for the inter-view prediction is notreordered. In this case, a reference picture list for temporalprediction is generated only. And, the inter-view prediction referencepicture list reordering unit 970 reorders reference pictures used forinter-view prediction. This is explained in detail with reference toFIG. 16 as follows.

FIG. 16 is an internal block diagram of the reference picture listreordering unit 970 for inter-view prediction according to oneembodiment of the present invention.

Referring to FIG. 16, the reference picture list reordering unit forinter-view prediction 970 includes a slice type checking unit 642, afifth identification information obtaining unit 971, a fifth referenceindex assignment changing unit 972, a sixth identification informationobtaining unit 973, and a sixth reference index assignment changing unit974.

The slice type checking unit 642 checks a slice type of a current slice.If so, it is then decided whether to execute reordering of a referencepicture list 0 and/or a reference picture list 1 according to the slicetype. Details of the slice type checking unit 642 can be inferred fromFIG. 10, which are omitted in the following description.

Each of the fifth and sixth identification information obtaining units971 and 973 obtains identification information indicating a referenceindex assigning method. And, each of the fifth and sixth reference indexassignment changing units 972 and 974 changes a reference index assignedto each reference picture of the reference picture list 0 and/or 1. Inthis case, the reference index can mean a view number of a referencepicture only. And, the identification information indicating thereference index assigning method may be flag information. For instance,if the flag information is true, an assignment of a view number ischanged. If the flag information is false, a reordering process of aview number can be terminated. If the flag information is true, each ofthe fifth and sixth reference index assignment changing units 972 and974 can obtain a view number difference according to the flaginformation. In this case, the view number difference means a differencebetween a view number of a current picture and a view number of apredicted picture. And, the view number of the predicted picture maymean a view number of a reference picture assigned right before. It isthen able to change view number assignment using the view numberdifference. In this case, the view number difference can beadded/subtracted to/from the view number of the predicted pictureaccording to the identification information.

Thus, to manage the reference picture list for the inter-view predictionas a separate reference picture list, it is necessary to newly define asyntax structure. As one embodiment of the contents explained in FIG. 15and FIG. 16, the syntax is explained with reference to FIG. 17, FIG. 18,and FIG. 19 as follows.

FIG. 17 and FIG. 18 are diagrams of syntax for reference picture listreordering according to one embodiment of the present invention.

Referring to FIG. 17, an operation of the reference picture listreordering unit for the temporal prediction 910 shown in FIG. 15 isrepresented as syntax. Compared to the blocks shown in FIG. 15, theslice type checking unit 642 corresponds to S1 and S6 and the fourthidentification information obtaining unit 940 corresponds to S7. Theinternal blocks of the third reference index assignment changing unit930 correspond to S3, S4, and S5, respectively. And, the internal blocksof the fourth reference index assignment changing unit 950 correspond toS8, S9, and S10, respectively.

Referring to FIG. 18, operations of the NAL type checking unit 960 andthe inter-view reference picture list reordering unit 970 arerepresented as syntax. Compared to the respective blocks shown in FIG.15 and FIG. 16, the NAL type checking unit 960 corresponds to S11, theslice type checking unit 642 corresponds to S13 and S16, the fifthidentification information obtaining unit 971 corresponds to S14, andthe sixth identification information obtaining unit 973 corresponds toS17. The fifth reference index assignment changing unit 972 correspondsto S15 and the sixth reference index assignment changing unit 974corresponds to S18.

FIG. 19 is a diagram of syntax for reference picture list reorderingaccording to another embodiment of the present invention.

Referring to FIG. 19, operations of the NAL type checking unit 960 andthe inter-view reference picture list reordering unit 970 arerepresented as syntax. Compared to the respective blocks shown in FIG.15 and FIG. 16, the NAL type checking unit 960 corresponds to S21, theslice type checking unit 642 corresponds to S22 and S25, the fifthidentification information obtaining unit 971 corresponds to S23, andthe sixth identification information obtaining unit 973 corresponds toS26. The fifth reference index assignment changing unit 972 correspondsto S24 and the sixth reference index assignment changing unit 974corresponds to S27.

As mentioned in the foregoing description, the reference picture listfor the inter-view prediction can be used by the inter-prediction unit700 and is usable for performing illumination compensation as well. Theillumination compensation is applicable in the course of performingmotion estimation/motion compensation. In case that a current pictureuses a reference picture in a different view, it is able to perform theillumination compensation more efficiently using the reference picturelist for the inter-view prediction. The illumination compensationsaccording to embodiments of the present invention are explained asfollows.

FIG. 20 is a diagram for a process for obtaining a illuminationdifference value of a current block according to one embodiment of thepresent invention.

Illumination compensation means a process for decoding an adaptivelymotion compensated video signal according to illumination change. And,it is applicable to a predictive structure of a video signal, forexample, inter-view prediction, intra-view prediction, and the like.

Illumination compensation means a process for decoding a video signalusing a illumination difference residual and a illumination differenceprediction value corresponding to a block to be decoded. In this case,the illumination difference prediction value can be obtained from aneighboring block of a current block. A process for obtaining aillumination difference prediction value from the neighboring block canbe decided using reference information for the neighbor block, and asequence and direction can be taken into consideration in the course ofsearching neighbor blocks. The neighboring block means an alreadydecoded block and also means a block decoded by considering redundancywithin the same picture for a view or time or a sequence decoded byconsidering redundancy within different pictures.

In comparing similarities between a current block and a candidatereference block, an illumination difference between the two blocksshould be taken into consideration. In order to compensate for theillumination difference, new motion estimation/compensation is executed.New SAD can be found using Formula 1.

$\begin{matrix}{{M_{cur} = {\frac{1}{S \times T}{\sum\limits_{i = m}^{m + S - 1}\; {\sum\limits_{j = n}^{n + T - 1}\; {f\left( {i,j} \right)}}}}}{{M_{ref}\left( {p,q} \right)} = {\frac{1}{S \times T}{\sum\limits_{i = p}^{p + S - 1}\; {\sum\limits_{j = q}^{q + T - 1}\; {r\left( {i,j} \right)}}}}}} & \left\lbrack {{Formula}\mspace{14mu} 1} \right\rbrack \\{{{NewSAD}\left( {x,y} \right)} = {\sum\limits_{i = m}^{m + S - 1}\; {\sum\limits_{j = n}^{n + T - 1}\; {\begin{matrix}{\left\{ {{f\left( {i,j} \right)} - M_{cur}} \right\} -} \\\begin{Bmatrix}{{r\left( {{i + x},{j + y}} \right)} -} \\{M_{ref}\left( {{m + x},{n + y}} \right)}\end{Bmatrix}\end{matrix}}}}} & \left\lbrack {{Formula}\mspace{14mu} 2} \right\rbrack\end{matrix}$

In this case, ‘Mcurr’ indicates an average pixel value of a currentblock and ‘Mref’ indicates an average pixel value of a reference block.‘f(i,j)’ indicates a pixel value of a current block and ‘r(i+x, j+y)’indicates a pixel value of a reference block. By performing motionestimation based on the new SAD according to the Formula 2, it is ableto obtain an average pixel difference value between the current blockand the reference block. And, the obtained average pixel differencevalue may be called an illumination difference value (IC_offset).

In case of performing motion estimation to which illuminationcompensation is applied, an illumination difference value and a motionvector are generated. And, the illumination compensation is executedaccording to Formula 3 using the illumination difference value and themotion vector.

$\begin{matrix}\begin{matrix}{{{NewR}\left( {i,j} \right)} = {\begin{Bmatrix}{{f\left( {i,j} \right)} -} \\M_{cur}\end{Bmatrix} - \begin{Bmatrix}{{r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)} -} \\{M_{{ref}\;}\left( {{m + x^{\prime}},{n + y^{\prime}}} \right)}\end{Bmatrix}}} \\{= {\begin{Bmatrix}{{f\left( {i,j} \right)} -} \\{r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}\end{Bmatrix} - \begin{Bmatrix}{M_{cur} - M_{{ref}\;}} \\\left( {{m + x^{\prime}},{n + y^{\prime}}} \right)\end{Bmatrix}}} \\{= {\begin{Bmatrix}{{f\left( {i,j} \right)} -} \\{r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}\end{Bmatrix} - {IC\_ offset}}}\end{matrix} & \left\lbrack {{Formula}\mspace{14mu} 3} \right\rbrack\end{matrix}$

In this case, NewR(i,j) indicates an illumination-compensated errorvalue (residual) and (x′, y′) indicates a motion vector.

An illumination difference value (Mcurr−Mref) should be transferred tothe decoding unit. The decoding unit carries out the illuminationcompensation in the following manner.

$\begin{matrix}\begin{matrix}{{f^{\prime}\left( {i, j} \right)} = {\begin{Bmatrix}{{{NewR}^{''}\left( {x^{\prime},y^{\prime},i,j} \right)} +} \\{r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}\end{Bmatrix} + \left\{ \begin{matrix}{M_{cur} - M_{ref}} \\\left( {{m + x^{\prime}},{n + y^{\prime}}} \right)\end{matrix} \right\}}} \\{= {\begin{Bmatrix}{{{NewR}^{''}\left( {x^{\prime},y^{\prime},i,j} \right)} +} \\{r\left( {{i + x^{\prime}},{j + y^{\prime}}} \right)}\end{Bmatrix} + {IC\_ offset}}}\end{matrix} & \left\lbrack {{Formula}\mspace{14mu} 4} \right\rbrack\end{matrix}$

In Formula 4, NewR″ (i,j) indicates a reconstructedillumination-compensated error value (residual) and f′ (i,j) indicates apixel value of a reconstructed current block.

In order to reconstruct a current block, an illumination differencevalue should be transferred to the decoding unit. And, the illuminationdifference value can be predicted from information of neighboringblocks. In order to further reduce a bit number to code the illuminationdifference value, it is able to send a difference value (RIC_offset)between the illumination difference value of the current block(IC_offset) and the illumination difference value of the neighboringblock (predIC_offset) only. This is represented as Formula 5.

RIC_offset=IC_offset−predIC_offset  [Formula 5]

FIG. 21 is a flowchart of a process for performing illuminationcompensation of a current block according to an embodiment of thepresent invention.

Referring t FIG. 21, first of all, an illumination difference value of aneighboring block indicating an average pixel difference value betweenthe neighboring block of a current block and a block referred to by theneighbor block is extracted from a video signal (S2110).

Subsequently, an illumination difference prediction value forillumination compensation of the current block is obtained using theillumination difference value (S2120). So, it is able to reconstruct anillumination difference value of the current block using the obtainedillumination difference prediction value.

In obtaining the illumination difference prediction value, it is able touse various methods. For instance, before the illumination differencevalue of the current block is predicted from the illumination differencevalue of the neighboring block, it is checked whether a reference indexof the current block is equal to that of the neighboring block. It isthen able to decide what kind of a neighboring block or a value will beused according to a result of the checking. For another instance, inobtaining the illumination difference prediction value, flag information(IC_flag) indicating whether to execute an illumination compensation ofthe current block can be used. And, flag information for the currentblock can be predicted using the information of the neighboring blocksas well. For another instance, it is able to obtain the illuminationdifference prediction value using both of the reference index checkingmethod and the flag information predicting method. These are explainedin detail with reference to FIGS. 22 to 24 as follows.

FIG. 22 is a block diagram of a process for obtaining an illuminationdifference prediction value of a current block using information for aneighbor block according to one embodiment of the present invention.

Referring to FIG. 22, it is able to use information for a neighboringblock in obtaining an illumination difference prediction value of acurrent block. In the present disclosure, a block can include amacroblock or a sub-macroblock. For instance, it is able to predict anillumination difference value of the current block using an illuminationdifference value of the neighboring block. Prior to this, it is checkedwhether a reference index of the current block is equal to that of theneighboring block. According to a result of the checking, it is thenable to decide what kind of a neighboring block or a value will be used.In FIG. 22, ‘refIdxLX’ indicates a reference index of a current block,‘refIdxLXN’ indicates a reference index of a block-N. In this case, ‘N’is a mark of a block neighbor to the current block and indicates A, B,or C. And, ‘PredIC_offsetN’ indicates an illumination difference valuefor illumination compensation of a neighbor block-N. If it is unable touse a block-C that is located at an upper right end of the currentblock, it is able to use a block-D instead of the block-C. Inparticular, information for the block-D is usable as information for theblock-C. If it is unable to use both of the block-B and the block-C, itis able to use a block-A instead. Namely, it is able to use theinformation for the block-A as the information for the block-B or theblock-C.

For another instance, in obtaining the illumination differenceprediction value, it is able to use flag information (IC_flag)indicating whether to execute an illumination compensation of thecurrent block. Alternatively, it is able to use both of the referenceindex checking method and the flag information predicting method inobtaining the illumination difference prediction value. In this case, ifthe flag information for the neighbor block indicates that theillumination compensation is not executed, i.e., if IC_falg==0, theillumination difference value ‘PredIC_offsetN’ of the neighbor block isset to 0.

FIG. 23 is a flowchart of a process for performing illuminationcompensation using information for a neighbor block according to oneembodiment of the present invention.

Referring to FIG. 23, the decoding unit extracts an average pixel valueof a reference block, a reference index of a current block, a referenceindex of the reference block, and the like from a video signal and isthen able to obtain an illumination difference prediction value of thecurrent block using the extracted information. The decoding unit obtainsa difference value (illumination difference residual) between anillumination difference value of the current block and the illuminationdifference prediction value and is then able to reconstruct anillumination difference value of the current block using the obtainedillumination difference residual and the illumination differenceprediction value. In this case, it is able to use information for aneighbor block to obtain the illumination difference prediction value ofthe current block. For instance, it is able to predict an illuminationdifference value of the current block using the illumination differencevalue of the neighbor block. Prior to this, it is checked whether areference index of the current block is equal to that of the neighborblock. According to a result of the checking, it is then able to decidewhat kind of a neighboring block or a value will be used.

In particular, an illumination difference value of a neighbor blockindicating an average pixel difference value between the neighbor blockof a current block and a block referred to by the neighbor block isextracted from a video signal (S2310).

Subsequently, it is checked whether a reference index of the currentblock is equal to a reference index of one of a plurality of neighborblocks (S2320).

As a result of the checking step S2320, if there exists at least oneneighbor block having the same reference index as that of the currentblock, it is checked whether there exist one corresponding neighborblock or not (S2325).

As a result of the checking step S2325, if there exists only oneneighbor block having the same reference index of the current block, anillumination difference value of the neighbor block having the samereference index of the current block is assigned to an illuminationdifference prediction value of the current block (S2330) In particular,it is ‘PredIC_offset=PredIC_offsetN’.

If the neighbor block having the same reference index as that of thecurrent block fails to exist as a result of the checking step S2320 orif there exist at least tow neighbor blocks having the same referenceindex as that of the current block as a result of the checking stepS2325, a median of illumination difference values (PredIC_offsetN, N=A,B, or C) of the neighbor blocks is assigned to an illuminationdifference prediction value of the current block (S650). In particular,it is ‘PredIC_offset=Median(PredIC_offsetA, PredIC_offsetB,PredIC_offsetC)’.

FIG. 24 is a flowchart of a process for performing illuminationcompensation using information for a neighbor block according to anotherembodiment of the present invention.

Referring to FIG. 24, a decoding unit has to reconstruct an illuminationdifference value of a current block to carry out illuminationcompensation. In this case, it is able to use information for a neighborblock to obtain an illumination difference prediction value of thecurrent block. For instance, it is able to predict an illuminationdifference value of the current block using the illumination differencevalue of the neighbor block. Prior to this, it is checked whether areference index of the current block is equal to that of the neighborblock. According to a result of the checking, it is then able to decidewhat kind of a neighboring block or a value will be used.

In particular, an illumination difference value of a neighbor blockindicating an average pixel difference value between the neighbor blockof a current block and a block referred to by the neighbor block isextracted from a video signal (S2410).

Subsequently, it is checked whether a reference index of the currentblock is equal to a reference index of one of a plurality of neighborblocks (S2420).

As a result of the checking step S720, if there exists at least oneneighbor block having the same reference index as that of the currentblock, it is checked whether there exist one corresponding neighborblock or not (S2430).

As a result of the checking step S2430, if there exists only oneneighbor block having the same reference index as that of the currentblock, an illumination difference value of the neighbor block having thesame reference index as that of the current block is assigned to anillumination difference prediction value of the current block (S2440).In particular, it is ‘PredIC_offset=PredIC_offsetN’.

If the neighbor block having the same reference index as that of thecurrent block fails to exist as a result of the checking step S2420, theillumination difference prediction value of the current block is set to0 (S2460). In particular, it is ‘PredIC_offset=0’.

If there exist at least two neighbor blocks having the same referenceindex as that of the current block as a result of the checking stepS2430, the neighbor block having a reference index different from thatof the current block is set to 0 and a median of illumination differencevalues of the neighbor blocks including the value set to 0 is assignedto the illumination difference prediction value of the current block(S2450). In particular, it is ‘PredIC_offset=Median(PredIC_offsetA,PredIC_offsetB, PredIC_offsetC)’. Yet, in case that there exists theneighbor block having the reference index different from that of thecurrent block, the value ‘0’ can be included in PredIC_offsetA,PredIC_offsetB, or PredIC_offsetC.

Meanwhile, view information for identifying a view of a picture and areference picture list for inter-view prediction are applicable tosynthesizing a picture in a virtual view. In a process for synthesizinga picture in a virtual view, a picture in a different view may bereferred to. So, if the view information and the reference picture listfor the inter-view prediction are used, it is able to synthesize apicture in a virtual view more efficiently. In the followingdescription, methods of synthesizing a picture in a virtual viewaccording to embodiments of the present invention are explained.

FIG. 25 is a block diagram of a process for predicting a current pictureusing a picture in a virtual view according to one embodiment of thepresent invention.

Referring to FIG. 25, in performing inter-view prediction in multi-viewvideo coding, it is able to predict a current picture using a picture ina view different from that of the current view as a reference picture.Yet, a picture in a virtual view is obtained using pictures in a viewneighbor to that of a current picture and the current picture is thenpredicted using the obtained picture in the virtual view. If so, theprediction can be more accurately performed. In this case, a viewidentifier indicating a view of a picture can be used to utilizepictures in neighbor views or pictures in a specific view. In case thatthe virtual view is generated, there must exist specific syntax forindicating whether to generate the virtual view. If the syntax indicatesthat the virtual view shall be generated, it is able to generate thevirtual view using the view identifier. The pictures in the virtual viewobtained by the view synthesis prediction unit 740 are usable asreference pictures. In this case, the view identifier can be assigned tothe pictures in the virtual view. In a process for performing motionvector prediction to transfer a motion vector, neighbor blocks of acurrent block can refer to the pictures obtained by the view synthesisprediction unit 740. In this case, to use the picture in the virtualview as the reference picture, a view identifier indicating a view of apicture can be utilized.

FIG. 26 is a flowchart of a process for synthesizing a picture of avirtual view in performing inter-view prediction in MVC according to anembodiment of the present invention.

Referring to FIG. 26, a picture in a virtual view is synthesized usingpictures in a view neighbor to that of a current picture. The currentpicture is then predicted using the synthesized picture in the virtualview. If so, it is able to achieve more accurate prediction. In casethat a picture in a virtual view is synthesized, there exists specificsyntax indicating whether to execute a prediction of a current pictureby synthesizing the picture in the virtual view. If it is decidedwhether to execute the prediction of the current picture, more efficientcoding is possible. The specific syntax is defined as an inter-viewsynthesis prediction identifier, which is explained as follows. Forinstance, a picture in a virtual view is synthesized by a slice layer todefine ‘view_synthesize_pred_flag’ indicating whether to execute aprediction of a current picture. And, a picture in a virtual view issynthesized by a macroblock layer to define ‘view_syn_pred_flag’indicating whether to execute a prediction of a current picture. If‘view_synthesize_pred_flag=1’, a current slice synthesizes a slice in avirtual view using a slice in a view neighbor to that of the currentslice. It is then able to predict the current slice using thesynthesized slice. If ‘view_synthesize_pred_flag=0’, a slice in avirtual view is not synthesized. Likewise, if ‘view_syn_pred_flag=1’, acurrent macroblock synthesizes a macroblock in a virtual view using amacroblock in a view neighbor to that of the current macroblock. It isthen able to predict the current macroblock using the synthesizedmacroblock. If ‘view_syn_pred_flag=0’, a macroblock in a virtual view isnot synthesized. Hence, in the present invention, the inter-viewsynthesis prediction identifier indicating whether to obtain a picturein a virtual view is extracted from a video signal. It is then able toobtain the picture in the virtual view using the inter-view synthesisprediction identifier.

As mentioned in the foregoing description, view information foridentifying a view of a picture and a reference picture list forinter-view prediction can be used by the inter-prediction unit 700. And,they can be used in performing weighted prediction as well. The weightedprediction is applicable to a process for performing motioncompensation. In doing so, if a current picture uses a reference picturein a different view, it is able to perform the weighted prediction moreefficiently using the view information and the reference picture listfor the inter-view prediction. Weighted prediction methods according toembodiments of the present invention are explained as follows.

FIG. 27 is a flowchart of a method of executing weighted predictionaccording to a slice type in video signal coding according to thepresent invention.

Referring to FIG. 27, weighted prediction is a method of scaling asample of motion compensated prediction data within a P-slice or B-slicemacroblock. A weighted prediction method includes an explicit mode forperforming weighted prediction for a current picture using a weightedcoefficient information obtained from information for reference picturesand an implicit mode for performing weighted prediction for a currentpicture using a weighted coefficient information obtained frominformation for a distance between the current picture and one ofreference pictures. The weighted prediction method can be differentlyapplied according to a slice type of a current macroblock. For instance,in the explicit mode, the weighted coefficient information can be variedaccording to whether a current macroblock, on which weighted predictionis performed, is a macroblock of a P-slice or a macroblock of a B-slice.And, the weighted coefficient of the explicit mode can be decided by anencoder and can be transferred by being included in a slice header. Onthe other hand, in the implicit mode, a weighted coefficient can beobtained based on a relatively temporal position of List 0 and List 1.For instance, if a reference picture is temporarily close to a currentpicture, a great weighted coefficient is applicable. If a referencepicture is temporarily distant from to a current picture, a smallweighted coefficient is applicable.

First of all, a slice type of a macroblock to apply weighted predictionthereto is extracted from a video signal (S2710).

Subsequently, weighted prediction can be performed on a macroblockaccording to the extracted slice type (S2720).

In this case, the slice type can include a macroblock to whichinter-view prediction is applied. The inter-view prediction means that acurrent picture is predicted using information for a picture in a viewdifferent from that of the current picture. For instance, the slice typecan include a macroblock to which temporal prediction for performingprediction using information for a picture in a same view as that of acurrent picture is applied, a macroblock to which the inter-viewprediction is applied, and a macroblock to which both of the temporalprediction and the inter-view prediction are applied. And, the slicetype can include a macroblock to which temporal prediction is appliedonly, a macroblock to which inter-view prediction is applied only, or amacroblock to which both of the temporal prediction and the inter-viewprediction are applied. Moreover, the slice type can include two of themacroblock types or all of the three macroblock types. This will beexplained in detail with reference to FIG. 28 later. Thus, in case thata slice type including a inter-view prediction applied macroblock isextracted from a video signal, weighted prediction is performed usinginformation for a picture in a view different from that of a currentpicture. In doing so, a view identifier for identifying a view of apicture can be utilized to use information for a picture in a differentview.

FIG. 28 is a diagram of macroblock types allowable in a slice type invideo signal coding according to one embodiment of the presentinvention.

Referring to FIG. 28, if a P-slice type by inter-view prediction isdefined as VP (View_P), an intra-macroblock I, a macroblock P predictedfrom one picture in a current view, or a macroblock VP predicted fromone picture in a different view is allowable for the P-slice type by theinter-view prediction (2810).

In case that a B-slice type by inter-view prediction is defined as VB(View_B), a macroblock P or B predicted from at least one picture in acurrent view or a macroblock VP or VB predicted from at least onepicture in a different view is allowable (2820).

In case that a slice type, on which prediction is performed usingtemporal prediction, inter-view prediction, or both of the temporalprediction and the inter-view prediction, is defined as ‘Mixed’, anintra-macroblock I, a macroblock P or B predicted from at least onepicture in a current view, a macroblock VP or VB predicted from at leastone picture in a different view, or a macroblock ‘Mixed’ predicted usingboth of the picture in the current view and the picture in the differentview is allowable for the mixed slice type (2830). In this case, inorder to use the picture in the different view, it is able to use a viewidentifier for identifying a view of a picture.

FIG. 29 and FIG. 30 are diagrams of syntax for executing weightedprediction according to a newly defined slice type according to oneembodiment of the present invention.

As mentioned in the foregoing description of FIG. 28, if the slice typeis decided as VP, VB, or Mixed, the syntax for performing theconventional weighted prediction (e.g., H.264) can be modified into FIG.29 or FIG. 30.

For instance, if a slice type is P-slice by temporal prediction, a part‘if(slice_type !=VP∥slice_type !=VB)’ is added (2910).

If a slice type is a B-slice by temporal prediction, the if-statementcan be modified into ‘if(slice_type==B∥slice_type==Mixed)’ (2920).

By newly defining a VP slice type and a VB slice type, a format similarto FIG. 29 can be newly added (2930, 2940). In this case, sinceinformation for a view is added, syntax elements include ‘view’ parts,respectively. For example, there is ‘luma_log 2_view_weight_denom,chroma_log 2_view_weight_denom’.

FIG. 31 is a flowchart of a method of executing weighted predictionusing flag information indicating whether to execute inter-view weightedprediction in video signal coding according to the present invention.

Referring to FIG. 31, in video signal coding to which the presentinvention is applied, in case of using flag information indicatingwhether weighted prediction will be executed, more efficient coding isenabled.

The flag information can be defined based on a slice type. For instance,there can exist flag information indicating whether weighted predictionwill be applied to a P-slice or a SP-slice or flag informationindicating whether weighted prediction will be applied to a B-slice.

In particular, the flag information can be defined as‘weighted_pred_flag’ or ‘weighted_bipred_idc’. If‘weighted_pred_flag=0’, it indicates that weighted prediction is notapplied to the P-slice and the SP-slice. If ‘weighted_pred_flag=1’, itindicates that weighted prediction is applied to the P-slice and theSP-slice. If ‘weighted_bipred_idc=0’, it indicates that default weightedprediction is applied to the B-slice. If ‘weighted_bipred_idc=1’, itindicates that explicit weighted prediction is applied to the B-slice.If ‘weighted_bipred_idc=2’, it indicates that implicit weightedprediction is applied to the B-slice.

In multi-view video coding, flag information indicating whether weightedprediction will be executed using information for an inter-view picturecan be defined based on a slice type.

First of all, a slice type and flag information indicating whetherinter-view weighted prediction will be executed are extracted from avideo signal (S3110, S3120). In this case, the slice type can include amacroblock to which temporal prediction for performing prediction usinginformation for a picture in a same view as that of a current picture isapplied and a macroblock to which inter-view prediction for performingprediction using information for a picture in a view different from thatof a current picture is applied.

It is then able to decide a weighted prediction mode based on theextracted slice type and the extracted flag information (S3130).

Subsequently, it is able to perform weighted prediction according to thedecided weighted prediction mode (S3140). In this case, the flaginformation can include flag information indicating whether weightedprediction will be executed using information for a picture in a viewdifferent from that of a current picture as well as the aforesaid‘weighted_pred_flag’ and ‘weighted_bipred_flag’. This will be explainedin detail with reference to FIG. 32 later.

Hence, in case that a slice type of a current macroblock is a slice typeincluding a macroblock to which inter-view prediction is applied, moreefficient coding is enabled rather than a case of using flag informationindicating whether weighted prediction will be executed usinginformation for a picture in a different view.

FIG. 32 is a diagram to explain a weight predicting method according toflag information indicating whether to execute weighted prediction usinginformation for a picture in a view different from that if a currentpicture according to one embodiment of the present invention.

Referring to FIG. 32, for example, flag information indicating whetherweighted prediction will be executed using information for a picture ina view different from that of a current picture can be defined as‘view_weighted_pred_flag’ or ‘view_weighted bipred_flag’.

If ‘view_weighted_pred_flag=0’, it indicates that weighted prediction isnot applied to a VP-slice. If ‘view_weighted_pred_flag=1’, explicitweighted prediction is applied to a VP-slice. If‘view_weighted_bipred_flag=0’, it indicates that default weightedprediction is applied to a VB-slice. If ‘view weighted_bipred_flag=1’,it indicates that explicit weighted prediction is applied to a VB-slice.If ‘view_weighted_bipred_flag=2’, it indicates that implicit defaultweighted prediction is applied to a VB-slice.

In case that implicit weighted prediction is applied to a VB-slice, aweight coefficient can be obtained from a relative distance between acurrent view and a different view. In case that implicit weightedprediction is applied to a VB-slice, weighted prediction can beperformed using a view identifier identifying a view of a picture or apicture order count (POC) rendered by considering discrimination of eachview.

The above flag informations can be included in a picture parameter set(PPS). In this case, the picture parameter set (PPS) means headerinformation indicating an encoding mode of all pictures (e.g., entropyencoding mode, quantization parameter initial value by picture unit,etc.) Yet, the picture parameter set is not attached to all of thepictures. If a picture parameter set does not exist, a picture parameterset existing right before is used as header information.

FIG. 33 is a diagram of syntax for executing weighted predictionaccording to newly defined flag information according to one embodimentof the present invention.

Referring to FIG. 33, in multi-view video coding to which the presentinvention is applied, in case that a slice type including a macroblockapplied to inter-view prediction and flag information indicating whetherweighted prediction will be executed using information for a picture ina view different from that of a current picture are defined, it isnecessary to decide what kind of weighted prediction will be executedaccording to a slice type.

For instance, if a slice type, as shown in FIG. 33, extracted from avideo signal is a P-slice or a SP-slice, weighted prediction can beexecuted if ‘weighted_pred_flag=1’. In case that a slice type is aB-slice, weighted prediction can be executed if‘weighted_bipred_flag=1’. In case that a slice type is a VP-slice,weighted prediction can be executed if ‘view_weighted_pred_flag=1’. Incase that a slice type is a VB-slice, weighted prediction can beexecuted if ‘view_weighted_bipred_flag=1’.

FIG. 34 is a flowchart of a method of executing weighted predictionaccording to a NAL (network abstraction layer) unit according to anembodiment of the present invention.

Referring to FIG. 34, first of all, a NAL unit type (nal_unit_type) isextracted from a video signal (S910). In this case, the NAL unit typemeans an identifier indicating a type of a NAL unit. For instance, if‘nal_unit_type=5’, a NAL unit is a slice of an IDR picture. And, the IDR(instantaneous decoding refresh) picture means a head picture of a videosequence.

Subsequently, it is checked whether the extracted NAL unit type is a NALunit type for multi-view video coding (S3420).

If the NAL unit type is the NAL unit type for multi-view video coding,weighted prediction is carried out using information for a picture in aview different from that of a current picture (S3430). The NAL unit typecan be a NAL unit type applicable to both scalable video coding andmulti-view video coding or a NAL unit type for multi-view video codingonly. Thus, if the NAL unit type is for multi-view video coding, theweighted prediction should be executed using the information for thepicture in the view different from that of the current picture. So, itis necessary to define new syntax. This will be explained in detail withreference to FIG. 35 and FIG. 36 as follows.

FIG. 35 and FIG. 36 are diagrams of syntax for executing weightedprediction in case that a NAL unit type is for multi-view video codingaccording to one embodiment of the present invention.

First of all, if a NAL unit type is a NAL unit type for multi-view videocoding, syntax for executing conventional weighted prediction (e.g.,H.264) can be modified into the syntax shown in FIG. 35 or FIG. 36. Forinstance, a reference number 3510 indicates a syntax part for performingconventional weighted prediction and a reference number 3520 indicates asyntax part for performing weighted prediction in multi-view videocoding. So, the weighted prediction is performed by the syntax part 3520only if the NAL unit type is the NAL unit type for multi-view videocoding. In this case, since information for a view is added, each syntaxelement includes a ‘view’ portion. For instance, there is ‘luma_view_log2_weight_denom, chroma_view_log 2_weight_denom’ or the like. And, areference number 3530 in FIG. 36 indicates a syntax part for performingconventional weighted prediction and a reference number 3540 in FIG. 36indicates a syntax part indicates a syntax part for performing weightedprediction in multi-view video coding. So, the weighted prediction isperformed by the syntax part 3540 only if the NAL unit type is the NALunit type for multi-view video coding. Likewise, since information for aview is added, each syntax element includes a ‘view’ portion. Forinstance, there is ‘luma_view_weight_(—)11_flag,chroma_view_weight_(—)11_flag’ or the like. Thus, if a NAL unit type formulti-view video coding is defined, more efficient coding is enabled una manner of performing weighted prediction using information for apicture in a view different from that of a current picture.

FIG. 37 is a block diagram of an apparatus for decoding a video signalaccording to an embodiment of the present invention.

Referring to FIG. 37, an apparatus for decoding a video signal accordingto the present invention includes a slice type extracting unit 3710, aprediction mode extracting unit 3720 and a decoding unit 3730.

FIG. 38 is a flowchart of a method of decoding a video signal in thedecoding apparatus shown in FIG. 37 according to one embodiment of thepresent invention.

Referring to FIG. 38, a method of decoding a video signal according toone embodiment of the present invention includes a step S3810 ofextracting a slice type and a macroblock prediction mode, and a stepS3820 of decoding a current macroblock according to the slice typeand/or macroblock prediction mode.

First, a prediction scheme used by an embodiment of the presentinvention is explained to help in the understanding of the presentinvention. The prediction scheme may be classified into an intra-viewprediction (e.g., prediction between pictures in a same view) and aninter-view prediction (e.g., prediction between pictures in differentviews). And, the intra-view prediction can be the same prediction schemeas a general temporal prediction.

According to the present invention, the slice type extracting unit 3710extracts a slice type of a slice including a current macroblock (S3810).

In this case, a slice type field (slice_type) indicating a slice typefor intra-view prediction and/or a slice type field (view_slice_type)indicating a slice type for inter-view prediction may be provided aspart of the video signal syntax to provide the slice type. This will bedescribed in greater deal below with respect to FIGS. 6( a) and 6(b).And, each of the slice type (slice_type) for intra-view prediction andthe slice type (view_slice_type) for inter-view prediction may indicate,for example, an I-slice type (I_SLICE), a P-slice type (P_SLICE), or aB-slice type (B_SLICE).

For instance, if ‘slice_type’ of a specific slice is a B-slice and ‘viewslice_type’ is a P-slice, a macroblock in the specific slice is decodedby a B-slice (B_SLICE) coding scheme in an intra-view direction (i.e., atemporal direction) and/or by a P-slice (P_SLICE) coding scheme in aview direction.

Meanwhile, the slice type is able to include a P-slice type (VP) forinter-view prediction, a B-slice type (VB) for inter-view prediction anda mixed slice type (Mixed) by prediction resulting from mixing bothprediction types. Namely, the mixed slice type provides for predictionusing a combination of intra-view and inter-view prediction.

In this case, a P-slice type for inter-view prediction means a case thateach macroblock or macroblock partition included in a slice is predictedfrom one picture in a current view or one picture in a different view. AB-slice type for inter-view prediction means a case that each macroblockor macroblock partition included in a slice is predicted from ‘one ortwo pictures in a current view’ or ‘one picture in a different view ortwo pictures in different views, respectively’. And, a mixed slice typefor prediction resulting from mixing both predictions means a case thateach macroblock or macroblock partition included in a slice is predictedfrom ‘one or two pictures in a current view’, ‘one picture in adifferent view or two pictures in different views, respectively’, or‘one or two pictures in a current view and one picture in a differentview or two pictures in different views, respectively’.

In other words, a referred picture and allowed macroblock type differ ineach slice type, which will be explained in detail with reference toFIG. 43 and FIG. 44 later.

And, the syntax among the aforesaid embodiments of the slice type willbe explained in detail with reference to FIG. 40 and FIG. 41 later.

The prediction mode extracting unit 3720 may extract a macroblockprediction mode indicator indicating whether the current macroblock is amacroblock by intra-view prediction, a macroblock by inter-viewprediction or a macroblock by prediction resulting from mixing bothtypes of prediction (S3820). For this, the present invention defines amacroblock prediction mode (mb_pred_mode). One embodiment of themacroblock prediction modes will be explained in detail with referenceto FIGS. 39,40 and FIGS. 41 later.

The decoding unit 3730 decodes the current macroblock according to theslice type and/or the macroblock prediction mode to receive/produce thecurrent macroblock (S3820). In this case, the current macroblock can bedecoded according to the macroblock type of the current macroblockdecided from the macroblock type information. And, the macroblock typecan be decided according to the macroblock prediction mode and the slicetype.

In case that the macroblock prediction mode is a mode for intra-viewprediction, the macroblock type is decided according to a slice type forintra-view prediction and the current macroblock is then decoded byintra-view prediction according to the decided macroblock type.

In case that the macroblock prediction mode is a mode for inter-viewprediction, the macroblock type is decided according to a slice type forinter-view prediction and the current macroblock is then decoded by theinter-view prediction according to the decided macroblock type.

In case that the macroblock prediction mode is a mode for predictionresulting from mixing both predictions, the macroblock type is decidedaccording to a slice type for intra-view prediction and a slice type forinter-view prediction, and the current macroblock is then decoded by theprediction resulting from mixing both predictions according to each ofthe decided macroblock types.

In this case, the macroblock type depends on a macroblock predictionmode and a slice type. In particular, a prediction scheme to be used fora macroblock type may be determined from a macroblock prediction mode,and a macroblock type is then decided from macroblock type informationby a slice type according to the prediction scheme. Namely, one of orboth of the extracted slice_type and view_slice_type are selected basedon the macroblock prediction mode.

For instance, if a macroblock prediction mode is a mode for inter-viewprediction, a macroblock type may be decided from a macroblock table ofslice types (I, P, B) corresponding to a slice type (view_slice_type)for inter-view prediction. The relation between a macroblock predictionmode and a macroblock type will be explained in detail with reference toFIGS. 39,40 and FIGS. 41 later.

FIG. 39 is a diagram of a macroblock prediction modes according toexample embodiments of the present invention.

In FIG. 39( a), a table corresponding to one embodiment of macroblockprediction modes (mb_pred_mode) according to the present invention isshown.

In case that intra-view prediction, i.e., temporal prediction is usedfor a macroblock only, ‘0’ is assigned to a value of the ‘mb_pred_mode’.In case that inter-view prediction is used for a macroblock only, ‘1’ isassigned to a value of the ‘mb_pred_mode’. In case that both temporaland inter-view prediction is used for a macroblock, ‘2’ is assigned to avalue of the ‘mb_pred_mode’.

In this case, if a value of the ‘mb_pred_mode’ is ‘1’, i.e., if the‘mb_pred_mode’ indicates the inter-view prediction, view direction List0(ViewList0) or view direction List1 (ViewList1) is defined as areference picture list for the inter-view prediction.

In FIG. 39( b), the relation between a macroblock prediction mode and amacroblock type according to another embodiment is shown.

If a value of ‘mb_pred_mode’ is ‘0’, temporal prediction is used only.And, a macroblock type is decided according to a slice type (slice_type)for intra-view prediction.

If a value of ‘mb_pred_mode’ is ‘1’, inter-view prediction is used only.And, a macroblock type is decided according to a slice type (viewslice_type) for inter-view prediction.

If a value of ‘mb_pred_mode’ is ‘2’, mixed prediction of both temporaland intra-view prediction is used. And, two macroblock types are decidedaccording to a slice type (slice_type) for intra-view prediction and aslice type (view_slice_type) for inter-view prediction.

Based on the macroblock prediction mode, the macroblock type is givenbased on the slice type as shown in tables 1-3 below. [Please inserttables 7-12-7-14 in N6540 here as tables 1-3]

In other words, in this embodiment, a prediction scheme used for amacroblock and a slice type referred to are decided by a macroblockprediction mode. And, a macroblock type is decided according to theslice type.

FIG. 40 and FIG. 41 are diagrams of example embodiments of the syntax ofa portion of the video signal received by the apparatus for decoding thevideo signal. As shown, the syntax has slice type and macroblockprediction mode information according to an embodiment of the presentinvention.

In FIG. 40, an example syntax is shown. In the syntax, the field‘slice_type’ and the field ‘view slice type’ provide slice types and thefield ‘mb_pred_mode’ provides a macroblock prediction mode.

According to the present invention, the ‘slice_type’ field provides aslice type for intra-view prediction and the ‘view_slice_type’ fieldprovides a slice type for inter-view prediction. Each slice type canbecome I-slice type, P-slice type or B-slice type. If a value of the‘mb_pred_mode’ is ‘0’ or ‘1’, one macroblock type is decided. Yet, incase that a value of the ‘mb_pred_mode’ is ‘2’, it can be seen thatanother macroblock type (or two types) is further decided. In otherwords, the syntax shown in (a) of FIG. 40 indicates that‘view_slice_type’ is added to further apply the conventional slice types(I, P, B) to multi-view video coding.

In FIG. 41, another example syntax is shown. In the syntax, a‘slice_type’ field is employed to provide a slice type and a‘mb_pred_mode’ field is employed to provide a macroblock predictionmode.

According to the present invention, the ‘slice_type’ field may include,among others, a slice type (VP) for inter-view prediction, a slicetype-B (VB) for inter-view prediction and a mixed slice type (Mixed) forprediction resulting from mixing both intra-view and inter-viewpredictions.

If a value in the ‘mb_pred_mode’ field is ‘0’ or ‘1’, one macroblocktype is decided. Yet, in case that a value of the ‘mb_pred_mode’ fieldis ‘2’, it can be seen that an additional (i.e., total of two)macroblock type is decided. In this embodiment, the slice typeinformation exists in a slice header, which will be explained in detailwith respect to FIG. 42. In other words, the syntax shown in FIG. 41indicates that VP, VB and Mixed slice types are added to theconventional slice type (slice_type).

FIG. 42 are diagrams of examples for applying the slice types shown inFIG. 41.

The diagram in FIG. 42( a) shows that a P-slice type (VP) for inter-viewprediction, a B-slice type (VB) for inter-view prediction and a mixedslice type (Mixed) for prediction resulting from mixing both predictionsmay exist as the slice type, in addition to other slice types, in aslice header. In particular, the slice types VP, VB and Mixed accordingto an example embodiment are added to the slice types that may exist ina general slice header.

The diagram in FIG. 42( b) shows that a P-slice type (VP) for inter-viewprediction, a B-slice type (VB) for inter-view prediction and a mixedslice type (Mixed) for prediction resulting from mixing both predictionsmay exist as the slice type in a slice header for multi-view videocoding (MVC). In particular, the slice types according to an exampleembodiment are defined in a slice header for multi-view video coding.

The diagram in FIG. 42( c) shows that a slice type (VP) for inter-viewprediction, a B-slice type (VB) for inter-view prediction and a mixedslice type (Mixed) for prediction resulting from mixing both predictionsmay exist as the slice type, in addition to existing slice type forscalable video coding, in a slice header for scalable video coding(SVC). In particular, the slice types VP, VB and Mixed according to anexample embodiment are added to slice types that may exist in a sliceheader of the scalable video coding (SVC) standard.

FIG. 43 is a diagram of various slice type examples included in theslice type shown in FIG. 41.

In FIG. 43( a), a case that a slice type is predicted from one picturein a different view is shown. So, a slice type becomes a slice type (VP)for inter-view prediction.

In FIG. 43( b), a case that a slice type is predicted from two picturesin different views, respectively is shown. So, a slice type becomes aB-slice type (VB) for inter-view prediction.

In FIGS. 43( c) and 43(f), a case that a slice type is predicted fromone or two pictures in a current view and one picture in a differentview is shown. So, a slice type becomes a mixed slice type (Mixed) forprediction resulting from mixing both predictions. Also, in FIGS. 43( d)and 43(e), a case that a slice type is predicted from one or twopictures in a current view and two pictures in different views is shown.So, a slice type also becomes a mixed slice type (Mixed).

FIG. 44 is a diagram of a macroblock allowed for the slice types shownin FIG. 41.

Referring to FIG. 44, an intra macroblock (I), a macroblock (P)predicted from one picture in a current view or a macroblock (VP)predicted from one picture in a different view is allowed for a P-slicetype (VP) by inter-view prediction.

An intra macroblock (I), a macroblock (P or B) predicted from one or twopictures in a current view or a macroblock VP or VB predicted from onepicture in a different view or two pictures in different views,respectively, are allowed for a B-slice type (VB) by inter-viewprediction.

And, an intra macroblock (I); a macroblock (P or B) predicted from oneor two pictures in a current view; a macroblock (VP or VB) predictedfrom one picture in a different view or two pictures in different views,respectively, or a macroblock (Mixed) predicted from one or two picturesin a current view, one picture in a different view or two pictures indifferent views, respectively, are allowed for a mixed slice type(Mixed).

FIGS. 45-47 are diagrams of a macroblock type of a macroblock existingin a mixed slice type (Mixed) according to embodiments of the presentinvention.

In FIGS. 45( a) and 45(b), configuration schemes for a macroblock type(mb_type) and sub-macroblock type (sub_mb_type) of a macroblock existingin a mixed slice are shown, respectively.

In FIGS. 46 and 47, binary representation of predictive direction(s) ofa macroblock existing in a mixed slice and actual predictivedirection(s) of the mixed slice are shown, respectively.

According to an embodiment of the present invention, a macroblock type(mb_type) is prepared by considering both a size (Partition_Size) of amacroblock partition and a predictive direction (Direction) of amacroblock partition.

And, a sub-macroblock type (sub_mb_type) is prepared by considering botha size (Sub_Partition_Size) of a sub-macroblock partition and apredictive direction (Sub_Direction) of each sub-macroblock partition.

Referring to FIG. 45( a), ‘Direction0’ and ‘Direction1’ indicate apredictive direction of a first macroblock partition and a predictivedirection of a second macroblock partition, respectively. In particular,in case of a 8×16 macroblock, ‘Direction0’ indicates a predictivedirection for a left 8×16 macroblock partition and ‘Direction1’indicates a predictive direction for a right 8×16 macroblock partition.A configurational principle of macroblock type (mb_type) is explained indetail as follows. First, the first two bits indicate a partition size(Partition_Size) of a corresponding macroblock and a value of 0˜3 isavailable for the first two bits. And, four bits following the first twobits indicate a predictive direction (Direction) in case that amacroblock is divided into partitions.

For instance, in case of a 16×16 macroblock, four bits indicating apredictive direction of the macroblock are attached to a rear of thefirst two bits. In case of a 16×8 macroblock, four bits following thefirst two bits indicate a predictive direction (Direction0) of a firstpartition and another four bits are attached to the former four bits toindicate a predictive direction (Direction1) of a second partition.Likewise, in case of a 8×16 macroblock, eight bits are attached to arear of the first two bits. In this case, the first four bits of theeight bits attached to the first two bits indicate a predictivedirection of a first partition and a next four bits indicate apredictive direction of a second partition.

Referring to FIG. 45( b), a predictive direction (Sub_Direction) of asub-macroblock is used in a same manner as a predictive direction(Direction) of the macroblock partition shown in FIG. 45( a). Aconfiguration principle of sub-macroblock type (sub_mb_type) isexplained in detail as follows.

First, the first two bits indicate a partition size (Partition_Size) ofa corresponding macroblock and the second two bits, next to the formertwo bits, indicate a partition size (Sub_Partition_Size) of asub-macroblock of the corresponding macroblock. A value of 0˜3 isavailable for each of the first and second two bits. Subsequently, fourbits attached next to the second two bits indicate a predictivedirection (Sub_Direction) in case that a macroblock is divided intosub-macroblock partitions. For instance, if a size (Partition_Size) of apartition of a macroblock is 8×8 and if a size (Sub_Partition_Size) of apartition of a sub-macroblock is 4×8, the first two bits have a value of3, the second two bits have a value of 2, the first four bits next tothe second two bits indicate a predictive direction for a left 4×8 blockof two 4×8 blocks, and the second four bits next to the first four bitsindicate a predictive direction for a right 4×8 block.

Referring to FIG. 46, a predictive direction of a macroblock isconstructed with four bits. And, it can be seen that each binaryrepresentation becomes ‘1’ according to a case of referring to a pictureat the left (L), top (T), right (R) or bottom (B) position of a currentpicture.

Referring to FIG. 47, for example, in case that a predictive directionis top (T), a picture located at a top in a view direction of a currentpicture is referred to. In case that a predictive direction correspondsto all directions (LTRB), it can be seen that pictures in all directions(LTRB) of a current picture are referred to.

FIG. 48 is a block diagram of an apparatus for encoding a video signalaccording to an embodiment of the present invention.

Referring to FIG. 48, an apparatus for encoding a video signal accordingto an embodiment of the present invention. The apparatus includes amacroblock type deciding unit 4810, a macroblock generating unit 4820and an encoding unit 4830.

FIG. 49 is a flowchart of a method of encoding a video signal in theencoding apparatus shown in FIG. 48 according to an embodiment of thepresent invention.

Referring to FIG. 49, a method of encoding a video signal according toan embodiment of the present invention includes a step S4910 of decidinga first macroblock type for intra-view prediction and a secondmacroblock type for inter-view prediction, a step S4920 of generating afirst macroblock having the first macroblock type and a secondmacroblock having the second macroblock type, a step S4930 of generatinga third macroblock using the first and second macroblocks, and a stepS4940 of encoding a macroblock type of a current macroblock and amacroblock prediction mode.

According to the present invention, the macroblock type deciding unit4810 decides a first macroblock type for intra-view prediction and asecond macroblock type for inter-view prediction (S4910) as described indetail above.

Subsequently, the macroblock generating unit 4820 generates a firstmacroblock having the first macroblock type and a second macroblockhaving the second macroblock type (S4920) using well-known predictiontechniques, and then generates a third macroblock using the first andsecond macroblocks (S4930). In this case, the third macroblock isgenerated according to a mean value between the first and secondmacroblocks.

Finally, the encoding unit 4830 encodes a macroblock type (mb_type) of acurrent macroblock and a macroblock prediction mode (mb_pred_mode) ofthe current macroblock by comparing encoding efficiencies of the firstto third macroblocks (S4940).

In this case, there are various methods to measure the encodingefficiencies. In particular, a method using RD (rate-distortion) cost isused in this embodiment of the present invention. As is well-known, inthe RD cost method, a corresponding cost is calculated with twocomponents: an encoding bit number generated from encoding acorresponding block and a distortion value indicating an error from anactual sequence.

The first and second macroblock types may be decided in a manner ofselecting a macroblock type having a minimum value of theabove-explained RD cost. For instance, a macroblock type having aminimum value of the RD cost among macroblock types by intra-viewprediction is decided as the first macroblock type. And, a macroblocktype having a minimum value of the RD cost among macroblock types byinter-view prediction is decided as the second macroblock type.

In the step of encoding the macroblock type and the macroblockprediction mode, the macroblock type an prediction mode associated withthe one of the first and second macroblocks having the smaller RD costmay be selected. Subsequently, the RD cost of the third macroblock isdetermined. Finally, the macroblock type and macroblock prediction modeof the current macroblock are encoded by comparing the RD cost of theselected first or second macroblock and the RD cost of the thirdmacroblock to each other.

If the RD cost of the selected first or second macroblock is equal to orgreater than the RD cost of the third macroblock, the macroblock typebecomes a macroblock type corresponding to the selected first or secondmacroblock.

For instance, if the RD cost of the first macroblock is smaller thanthat of the second and third macroblocks, the current macroblock is setas the first macroblock type. And, the macroblock prediction mode (i.e.,intra-view) becomes a prediction scheme of a macroblock corresponding tothe RD cost.

For instance, if the RD cost of the second macroblock is smaller thanthat of the first and third macroblocks, an inter-view prediction schemeas a prediction scheme of the second macroblock becomes the macroblockprediction mode of the current macroblock.

Meanwhile, if the RD cost of the third macroblock is smaller than the RDcosts of the first and second macroblocks, macroblock types correspondto both the first and second macroblock types. In particular, intra-viewprediction and inter-view prediction macroblock types become macroblocktypes of the current macroblock. And, the macroblock prediction modebecomes a mixed prediction scheme resulting from mixing intra-view andinter-view predictions.

Accordingly, the present invention provides at least the followingeffect or advantage.

The present invention is able to exclude the redundancy informationbetween views due to various prediction schemes between views and suchinformation as slice types, macroblock types and macroblock predictionmodes; thereby enhancing performance of encoding/decoding efficiency.

INDUSTRIAL APPLICABILITY

While the present invention has been described and illustrated hereinwith reference to the preferred embodiments thereof, it will be apparentto those skilled in the art that various modifications and variationscan be made therein without departing from the spirit and scope of theinvention. Thus, it is intended that the present invention covers themodifications and variations of this invention that come within thescope of the appended claims and their equivalents.

1-10. (canceled)
 11. A method for decoding multi-view video data in a multi-view video stream with a decoding apparatus, comprising: receiving, with the decoding apparatus, the multi-view video stream, the multi-view video stream including a random access picture, the random access picture including a random access slice, the random access slice referencing only a slice corresponding to a same time and a different view of the random access picture; obtaining, with the decoding apparatus, a random access flag for inter-view prediction from the multi-view video stream, the random access flag indicating whether a type of picture is the random access picture; obtaining, with the decoding apparatus, initialization information of a reference picture list for the random access slice based on the random access flag, the initialization information representing view relationships between a plurality of views, the initialization information including view number information and view identification information; initializing, with the decoding apparatus, the reference picture list using the view number information and the view identification information; obtaining, with the decoding apparatus, modification information for the initialized reference picture list from the multi-view video stream, the modification information representing how to assign an inter-view reference index in the initialized reference picture list; determining, with the decoding apparatus, an assignment modification value for modifying the inter-view reference index in the initialized reference picture list according to the modification information; modifying, with the decoding apparatus, the initialized reference picture list for inter-view prediction using the determined assignment modification value; determining, with the decoding apparatus, a prediction value of a macroblock in the random access picture based on the modified reference picture list; and decoding, with the decoding apparatus, the macroblock using the prediction value, wherein the initialization information is obtained from an extension area of a sequence header in the multi-view video stream.
 12. The method of claim 11, wherein the view number information indicates a number of reference views of the random access picture, and the view identification information provides a view identifier of each reference view for the random access picture.
 13. The method of claim 11, wherein the multi-view video data includes video data of a base view independent of other views, the base view being a view decoded without using inter-view prediction.
 14. The method of claim 11, wherein the inter-view reference index is assigned by performing a subtraction operation or an addition operation according to the modification information.
 15. The method of claim 11, wherein the modification information is obtained from a slice header.
 16. The method of claim 11, wherein the determined assignment modification value is used to assign the inter-view reference index to the random access picture in the initialized reference picture list.
 17. The method of claim 11, wherein the assignment modification value represents a variable associated with a view identifier of an inter-view reference picture in the initialized reference picture list.
 18. The method of claim 16, wherein the modifying step shifts other remaining pictures in the initialized reference picture list to later positions in the initialized reference picture list.
 19. An apparatus for decoding multi-view video data in a multi-view video stream, comprising: a parsing unit configured to receive the multiview video stream, the multi-view video stream including a random access picture, the random access picture including a random access slice, the random access slice referencing only a slice corresponding to a same time and a different view of the random access picture, the parsing unit configured to obtain a random access flag for inter-view prediction from the multi-view video stream, the random access flag indicating whether a type of picture is the random access picture, and the parsing unit configured to obtain initialization information of a reference picture list for the random access slice based on the random access flag, the initialization information representing view relationships between a plurality of views, the initialization information including view number information and view identification information; a decoded picture buffer unit configured to initialize the reference picture list using the view number information and the view identification information, and obtain modification information for the initialized reference picture list from the multi-view video stream, the modification information representing how to assign an inter-view reference index in the initialized reference picture list, the decoded picture buffer unit configured to determine an assignment modification value for modifying the inter-view reference index in the initialized reference picture list according to the modification information, and configured to modify the initialized reference picture list for inter-view prediction using the determined assignment modification value; and an inter-prediction unit configured to determine a prediction value of a macroblock in the random access picture based on the modified reference picture list, and configured to decode the macroblock using the prediction value, wherein the initialization information is obtained from an extension area of a sequence header in the multi-view video stream.
 20. The apparatus of claim 19, wherein the view number information indicates a number of reference views of the random access picture, and the view identification information provides a view identifier of each reference view for the random access picture.
 21. The apparatus of claim 19, wherein the multi-view video data includes video data of a base view independent of other views, the base view being a view decoded without using inter-view prediction.
 22. The apparatus of claim 19, wherein the inter-view reference index is assigned by performing a subtraction operation or an addition operation according to the modification information.
 23. The apparatus of claim 9, wherein the determined assignment modification value is used to assign the inter-view reference index to the random access picture in the initialized reference picture list.
 24. The apparatus of claim 9, wherein the assignment modification value represents a variable associated with a view identifier of an inter-view reference picture in the initialized reference picture list.
 25. The apparatus of claim 23, wherein the decoded picture buffer unit is configured to shift other remaining pictures in the initialized reference picture list to later positions in the initialized reference picture list. 